2020-12-24T07:16:57.202 INFO:root:teuthology version: 1.0.0-8e49dda4 2020-12-24T07:16:57.204 INFO:teuthology.results:subset: '25/100000' 2020-12-24T07:16:57.204 INFO:teuthology.results:seed: '4052' 2020-12-24T11:12:18.829 INFO:root:teuthology version: 1.0.0-8e49dda4 2020-12-24T11:12:18.858 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-12-24T23:03:43.299 INFO:teuthology.results:Tests finished! gathering results... 2020-12-24T23:05:14.066 INFO:scrape:Found 367 jobs 2020-12-24T23:05:23.543 INFO:scrape:Found 10 distinct failure reasons 2020-12-24T23:05:23.544 INFO:scrape:Crash: "2020-12-24T11:14:17.099064+0000 mon.b (mon.0) 364 : cluster [WRN] Health check failed: 1 daemons have recently crashed (RECENT_CRASH)" in cluster log 2020-12-24T23:05:23.545 INFO:scrape:ceph version 16.0.0-8560-g6e59b682 (6e59b6825a70a22f08c16a04dc1e16bb4393788a) pacific (dev) 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980) [0x7ff2bf2cf980] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x19c) [0x563ef8e0463c] 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x563ef8e047c6] 6: (PG::scrub(unsigned int, ThreadPool::TPHandle&)+0x195) [0x563ef8f3f365] 7: (ceph::osd::scheduler::PGScrub::run(OSD*, OSDShard*, boost::intrusive_ptr&, ThreadPool::TPHandle&)+0x1a) [0x563ef90ee52a] 8: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xcd5) [0x563ef8ea9905] 9: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x4ac) [0x563ef951fbdc] 10: (ShardedThreadPool::WorkThreadSharded::entry()+0x10) [0x563ef9522e90] 11: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7ff2bf2c46db] 12: clone() 2020-12-24T23:05:23.546 INFO:scrape:1 jobs: ['5735350'] 2020-12-24T23:05:23.547 INFO:scrape:suites: ['mon_election/connectivity', 'msgr-failures/many', 'msgr/async', 'objectstore/bluestore-comp-zstd', 'rados', 'rados/singleton/{all/pg-autoscaler', 'supported-random-distro$/{ubuntu_latest}}'] 2020-12-24T23:05:23.548 INFO:scrape: 2020-12-24T23:05:23.549 INFO:scrape:Dead: b'2020-12-24T22:09:20.781 DEBUG:teuthology.exit:Finished running handlers' 2020-12-24T23:05:23.549 INFO:scrape:4 jobs: ['5735272', '5735100', '5735363', '5735134'] 2020-12-24T23:05:23.550 INFO:scrape:suites intersection: ['clusters/{fixed-2', 'mon_election/connectivity', 'openstack}', 'rados', 'supported-random-distro$/{ubuntu_latest}', 'thrashosds-health'] 2020-12-24T23:05:23.551 INFO:scrape:suites union: ['1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{more-active-recovery}', '2-recovery-overrides/{more-partial-recovery}', 'backoff/normal', 'ceph', 'clusters/{fixed-2', 'crc-failures/default', 'd-balancer/on', 'fast/normal', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr/async-v1only', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/thrash-erasure-code-isa/{arch/x86_64', 'rados/thrash-erasure-code/{ceph', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'recovery-overrides/{more-active-recovery}', 'recovery-overrides/{more-partial-recovery}', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/mapgap', 'thrashers/morepggrow', 'thrashosds-health', 'workloads/ec-rados-plugin=isa-k=2-m=1}', 'workloads/ec-small-objects-fast-read}', 'workloads/pool-snaps-few-objects}', 'workloads/snaps-few-objects-localized}'] 2020-12-24T23:05:23.552 INFO:scrape: 2020-12-24T23:05:23.553 INFO:scrape:Failure: "2020-12-24T09:43:16.603643+0000 mon.c (mon.0) 1236 : cluster [WRN] Health check failed: 1 daemons have recently crashed (RECENT_CRASH)" in cluster log 2020-12-24T23:05:23.553 INFO:scrape:2 jobs: ['5735180', '5735147'] 2020-12-24T23:05:23.554 INFO:scrape:suites intersection: ['mon_election/connectivity', 'openstack}', 'rados', 'thrashers/mapgap', 'thrashosds-health'] 2020-12-24T23:05:23.555 INFO:scrape:suites union: ['ceph', 'cluster/{12-osds', 'clusters/{fixed-2', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-hybrid', 'openstack}', 'rados', 'rados/thrash-erasure-code-big/{ceph', 'rados/thrash-erasure-code-isa/{arch/x86_64', 'recovery-overrides/{more-async-partial-recovery}', 'recovery-overrides/{more-partial-recovery}', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{rhel_8}', 'thrashers/mapgap', 'thrashosds-health', 'workloads/ec-rados-plugin=isa-k=2-m=1}', 'workloads/ec-rados-plugin=jerasure-k=4-m=2}'] 2020-12-24T23:05:23.556 INFO:scrape: 2020-12-24T23:05:23.557 INFO:scrape:Failure: reached maximum tries (180) after waiting for 180 seconds 2020-12-24T23:05:23.558 INFO:scrape:6 jobs: ['5735058', '5735192', '5735104', '5735152', '5735285', '5735252'] 2020-12-24T23:05:23.558 INFO:scrape:suites intersection: ['3-start-upgrade', '4-wait', 'fixed-2', 'rados/cephadm/upgrade/{1-start'] 2020-12-24T23:05:23.559 INFO:scrape:suites union: ['2-repo_digest/defaut', '2-repo_digest/repo_digest', '3-start-upgrade', '4-wait', 'distro$/{centos_latest}', 'distro$/{rhel_8.0}', 'distro$/{rhel_latest}', 'fixed-2', 'mon_election/classic}', 'mon_election/connectivity}', 'rados/cephadm/upgrade/{1-start'] 2020-12-24T23:05:23.559 INFO:scrape: 2020-12-24T23:05:23.560 INFO:scrape:Failure: Command failed on smithi071 with status 1: 'sudo docker kill -s 1 ceph-e4f52fc0-45cb-11eb-8f8d-001a4aab830c-osd.9' 2020-12-24T23:05:23.560 INFO:scrape:1 jobs: ['5735195'] 2020-12-24T23:05:23.560 INFO:scrape:suites: ['1-install/luminous', 'backoff/normal', 'ceph', 'clusters/{openstack', 'd-balancer/on', 'distro$/{ubuntu_18.04}', 'mon_election/connectivity', 'msgr-failures/osd-delay', 'rados', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size', 'thrashers/mapgap', 'thrashosds-health', 'three-plus-one}', 'workloads/cache-snaps}'] 2020-12-24T23:05:23.561 INFO:scrape: 2020-12-24T23:05:23.561 INFO:scrape:Failure: Command failed (workunit test rados/test.sh) on smithi116 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dfe599e6b366a7e2da2b63bb3a660399ccd07d63 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' 2020-12-24T23:05:23.562 INFO:scrape:3 jobs: ['5735277', '5735181', '5735326'] 2020-12-24T23:05:23.563 INFO:scrape:suites intersection: ['clusters/{fixed-2', 'mon_election/classic', 'msgr-failures/few', 'openstack}', 'rados'] 2020-12-24T23:05:23.563 INFO:scrape:suites union: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async', 'msgr/async-v1only', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-hybrid', 'openstack}', 'rados', 'rados/basic/{ceph', 'rados/verify/{centos_latest', 'supported-random-distro$/{centos_8}', 'tasks/rados_api_tests', 'tasks/rados_api_tests}', 'validater/valgrind}'] 2020-12-24T23:05:23.563 INFO:scrape: 2020-12-24T23:05:23.564 INFO:scrape:Timeout 120 running ceph --cluster ceph osd dump --format=json 2020-12-24T23:05:23.564 INFO:scrape:1 jobs: ['5735351'] 2020-12-24T23:05:23.565 INFO:scrape:suites: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{more-partial-recovery}', 'backoff/peering', 'ceph', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'd-balancer/crush-compat', 'mon_election/classic', 'msgr-failures/few', 'msgr/async-v2only', 'objectstore/bluestore-comp-zstd', 'openstack}', 'rados', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/mapgap', 'thrashosds-health', 'workloads/cache-pool-snaps}'] 2020-12-24T23:05:23.566 INFO:scrape: 2020-12-24T23:05:23.567 INFO:scrape:Failure: Command failed (workunit test cephadm/test_cephadm.sh) on smithi052 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dfe599e6b366a7e2da2b63bb3a660399ccd07d63 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' 2020-12-24T23:05:23.567 INFO:scrape:1 jobs: ['5735273'] 2020-12-24T23:05:23.568 INFO:scrape:suites: ['mon_election/connectivity', 'rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_cephadm}'] 2020-12-24T23:05:23.568 INFO:scrape: 2020-12-24T23:05:23.568 INFO:scrape:Failure: Command failed (workunit test cephtool/test.sh) on smithi061 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dfe599e6b366a7e2da2b63bb3a660399ccd07d63 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh' 2020-12-24T23:05:23.569 INFO:scrape:1 jobs: ['5735315'] 2020-12-24T23:05:23.570 INFO:scrape:suites: ['mon_election/classic', 'msgr-failures/many', 'msgr/async-v1only', 'objectstore/bluestore-comp-lz4', 'rados', 'rados/singleton-bluestore/{all/cephtool', 'supported-random-distro$/{rhel_8}}'] 2020-12-24T23:05:23.571 INFO:scrape: 2020-12-24T23:05:23.571 INFO:scrape:Failure: Command failed (workunit test cephadm/test_adoption.sh) on smithi007 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dfe599e6b366a7e2da2b63bb3a660399ccd07d63 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_adoption.sh' 2020-12-24T23:05:23.572 INFO:scrape:1 jobs: ['5735101'] 2020-12-24T23:05:23.572 INFO:scrape:suites: ['mon_election/connectivity', 'rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_adoption}'] 2020-12-24T23:05:23.573 INFO:scrape: 2020-12-24T23:05:23.584 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/ failed: 17 dead: 4 running: 0 waiting: 0 queued: 0 passed: 346 Fail ================================================================= [5735058] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{rhel_latest} fixed-2 mon_election/classic} ----------------------------------------------------------------- time: 00:17:01 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735058/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735058/ sentry: https://sentry.ceph.com/organizations/ceph/?query=06ff3fe359434f179a91ad99acf5f0d6 reached maximum tries (180) after waiting for 180 seconds [5735101] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/connectivity task/test_adoption} ----------------------------------------------------------------- time: 00:15:00 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735101/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735101/ sentry: https://sentry.ceph.com/organizations/ceph/?query=12e06688e1214ffeb2f5d025fa9e7553 Command failed (workunit test cephadm/test_adoption.sh) on smithi007 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dfe599e6b366a7e2da2b63bb3a660399ccd07d63 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_adoption.sh' [5735104] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{rhel_8.0} fixed-2 mon_election/connectivity} ----------------------------------------------------------------- time: 00:18:44 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735104/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735104/ sentry: https://sentry.ceph.com/organizations/ceph/?query=01d30009d3c9485c85d231b9e821e413 reached maximum tries (180) after waiting for 180 seconds [5735147] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:30:00 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735147/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735147/ "2020-12-24T09:34:46.122324+0000 mon.b (mon.0) 2649 : cluster [WRN] Health check failed: 1 daemons have recently crashed (RECENT_CRASH)" in cluster log [5735152] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{centos_latest} fixed-2 mon_election/classic} ----------------------------------------------------------------- time: 00:17:41 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735152/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735152/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5deda4fd60e84390b952b6655367d5b5 reached maximum tries (180) after waiting for 180 seconds [5735180] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} ----------------------------------------------------------------- time: 00:30:39 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735180/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735180/ "2020-12-24T09:43:16.603643+0000 mon.c (mon.0) 1236 : cluster [WRN] Health check failed: 1 daemons have recently crashed (RECENT_CRASH)" in cluster log [5735181] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados tasks/rados_api_tests validater/valgrind} ----------------------------------------------------------------- time: 06:46:21 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735181/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735181/ sentry: https://sentry.ceph.com/organizations/ceph/?query=91ab5f02328c4291973323f1b45316a2 Command failed (workunit test rados/test.sh) on smithi063 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dfe599e6b366a7e2da2b63bb3a660399ccd07d63 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 ALLOW_TIMEOUTS=1 adjust- ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [5735192] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{centos_latest} fixed-2 mon_election/connectivity} ----------------------------------------------------------------- time: 00:17:17 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735192/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735192/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5e838f15f48c48baba098c55214b1d45 reached maximum tries (180) after waiting for 180 seconds [5735195] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/mapgap thrashosds-health workloads/cache-snaps} ----------------------------------------------------------------- time: 00:28:55 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735195/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735195/ Command failed on smithi071 with status 1: 'sudo docker kill -s 1 ceph-e4f52fc0-45cb-11eb-8f8d-001a4aab830c-osd.9' [5735252] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{rhel_latest} fixed-2 mon_election/classic} ----------------------------------------------------------------- time: 00:16:40 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735252/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735252/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1fe330bc75ac49999570792afe35f3b8 reached maximum tries (180) after waiting for 180 seconds [5735273] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:14:46 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735273/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735273/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b32ce36411d6400da3c9bc718e8f7234 Command failed (workunit test cephadm/test_cephadm.sh) on smithi052 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dfe599e6b366a7e2da2b63bb3a660399ccd07d63 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [5735277] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:29:58 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735277/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735277/ sentry: https://sentry.ceph.com/organizations/ceph/?query=58fd32ce79604914b2365afaf9849f7c Command failed (workunit test rados/test.sh) on smithi116 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dfe599e6b366a7e2da2b63bb3a660399ccd07d63 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [5735285] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{centos_latest} fixed-2 mon_election/connectivity} ----------------------------------------------------------------- time: 00:17:10 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735285/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735285/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f4bdcefaac2a45abb3b4ca26b599755d reached maximum tries (180) after waiting for 180 seconds [5735315] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:32:35 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735315/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735315/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f6a3e9b2d28b4a809ca3f11847bb4d42 Command failed (workunit test cephtool/test.sh) on smithi061 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dfe599e6b366a7e2da2b63bb3a660399ccd07d63 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh' [5735326] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} ----------------------------------------------------------------- time: 06:46:17 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735326/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735326/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5f6c88f9e4e74a23beecec0ebf59e878 Command failed (workunit test rados/test.sh) on smithi176 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dfe599e6b366a7e2da2b63bb3a660399ccd07d63 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 ALLOW_TIMEOUTS=1 adjust- ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [5735350] rados/singleton/{all/pg-autoscaler mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:15:22 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735350/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735350/ "2020-12-24T11:14:17.099064+0000 mon.b (mon.0) 364 : cluster [WRN] Health check failed: 1 daemons have recently crashed (RECENT_CRASH)" in cluster log [5735351] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps} ----------------------------------------------------------------- time: 01:47:15 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735351/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735351/ Command failed on smithi184 with status 124: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' Dead ================================================================= [5735100] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735100/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735100/ [5735134] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/pool-snaps-few-objects} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735134/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735134/ [5735272] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/snaps-few-objects-localized} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735272/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735272/ [5735363] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735363/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735363/ Pass ================================================================= [5735018] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/classic start} time: 00:13:34 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735018/ [5735019] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/snaps-few-objects} time: 00:25:23 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735019/ [5735020] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/filestore-xfs supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:07:36 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735020/ [5735021] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:08:36 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735021/ [5735022] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} tasks/module_selftest} time: 00:27:05 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735022/ [5735023] rados/cephadm/smoke-roleless/{distro/ubuntu_latest start} time: 00:11:38 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735023/ [5735024] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:27:05 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735024/ [5735025] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:14:06 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735025/ [5735026] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-radosbench} time: 02:02:08 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735026/ [5735027] rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:39:37 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735027/ [5735028] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:17:11 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735028/ [5735029] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/classic task/test_cephadm_repos} time: 00:11:26 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735029/ [5735030] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/write_fadvise_dontneed} time: 00:16:50 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735030/ [5735031] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} tasks/rados_stress_watch} time: 00:25:13 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735031/ [5735032] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:29:33 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735032/ [5735033] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados tasks/mon_recovery validater/valgrind} time: 00:28:20 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735033/ [5735034] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/careful thrashosds-health workloads/rbd_cls} time: 00:20:24 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735034/ [5735035] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:16:56 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735035/ [5735036] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:31:24 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735036/ [5735037] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/many workloads/rados_mon_workunits} time: 00:39:01 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735037/ [5735038] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/connectivity start} time: 00:13:03 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735038/ [5735039] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:42:57 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735039/ [5735040] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:08:41 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735040/ [5735041] rados/objectstore/{backends/alloc-hint supported-random-distro$/{ubuntu_latest}} time: 00:13:38 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735041/ [5735042] rados/rest/{mgr-restful supported-random-distro$/{centos_8}} time: 00:17:34 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735042/ [5735043] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:15:10 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735043/ [5735044] rados/standalone/{mon_election/classic supported-random-distro$/{rhel_8} workloads/crush} time: 00:18:50 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735044/ [5735045] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:33 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735045/ [5735046] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-octopus 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/classic thrashosds-health ubuntu_latest} time: 05:37:09 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735046/ [5735047] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:21:44 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735047/ [5735048] rados/cephadm/dashboard/{distro/centos_latest task/test_e2e} time: 00:31:51 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735048/ [5735049] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/admin_socket_objecter_requests} time: 00:42:19 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735049/ [5735050] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:24:26 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735050/ [5735051] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:21:49 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735051/ [5735052] rados/cephadm/orchestrator_cli/{2-node-mgr orchestrator_cli supported-random-distro$/{ubuntu_latest}} time: 00:11:19 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735052/ [5735053] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/classic start} time: 00:14:26 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735053/ [5735054] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:17:40 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735054/ [5735055] rados/cephadm/smoke-roleless/{distro/centos_8.0 start} time: 00:13:19 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735055/ [5735056] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-agent-big} time: 00:25:29 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735056/ [5735057] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:07:05 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735057/ [5735059] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/few rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:31:40 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735059/ [5735060] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8} tasks/progress} time: 00:23:48 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735060/ [5735061] rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:40:15 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735061/ [5735062] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:11:25 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735062/ [5735063] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:20:57 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735063/ [5735064] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 00:18:06 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735064/ [5735065] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:09:43 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735065/ [5735066] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} tasks/rados_striper} time: 00:10:08 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735066/ [5735067] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/default thrashosds-health workloads/snaps-few-objects} time: 00:38:29 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735067/ [5735068] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:17:26 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735068/ [5735069] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache-agent-small} time: 00:13:07 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735069/ [5735070] rados/standalone/{mon_election/connectivity supported-random-distro$/{centos_8} workloads/erasure-code} time: 00:44:18 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735070/ [5735071] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:13:14 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735071/ [5735072] rados/cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_orch_cli} time: 00:18:57 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735072/ [5735073] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:12:11 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735073/ [5735074] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados tasks/rados_api_tests validater/lockdep} time: 00:32:11 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735074/ [5735075] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/connectivity start} time: 00:13:34 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735075/ [5735076] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:29:12 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735076/ [5735077] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:16:30 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735077/ [5735078] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/one workloads/rados_mon_workunits} time: 00:36:44 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735078/ [5735079] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:46 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735079/ [5735080] rados/singleton/{all/test_envlibrados_for_rocksdb mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:22:48 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735080/ [5735081] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/mapgap thrashosds-health workloads/test_rbd_api} time: 00:19:04 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735081/ [5735082] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:21:06 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735082/ [5735083] rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:28:58 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735083/ [5735084] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8} tasks/dashboard} time: 01:12:38 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735084/ [5735085] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{centos_8}} time: 00:12:28 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735085/ [5735086] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/classic start} time: 00:15:20 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735086/ [5735087] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:16:46 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735087/ [5735088] rados/singleton/{all/thrash-backfill-full mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 01:16:29 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735088/ [5735089] rados/cephadm/smoke-roleless/{distro/centos_latest start} time: 00:12:46 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735089/ [5735090] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-pool-snaps} time: 00:20:44 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735090/ [5735091] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:09:00 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735091/ [5735092] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/connectivity start} time: 00:13:24 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735092/ [5735093] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} tasks/prometheus} time: 00:09:32 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735093/ [5735094] rados/standalone/{mon_election/classic supported-random-distro$/{ubuntu_latest} workloads/mgr} time: 00:16:03 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735094/ [5735095] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_big} time: 00:36:12 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735095/ [5735096] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:09:44 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735096/ [5735097] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:41:11 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735097/ [5735098] rados/singleton/{all/thrash-eio mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:30:47 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735098/ [5735099] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:05:39 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735099/ [5735102] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache-snaps-balanced} time: 00:25:17 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735102/ [5735103] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:10:30 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735103/ [5735105] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:09:04 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735105/ [5735106] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:27:52 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735106/ [5735107] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados tasks/rados_cls_all validater/valgrind} time: 02:24:25 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735107/ [5735108] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:26:54 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735108/ [5735109] rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:44:13 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735109/ [5735110] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:31:36 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735110/ [5735111] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:47 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735111/ [5735112] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:22:55 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735112/ [5735113] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:13:15 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735113/ [5735114] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/snaps-few-objects} time: 00:30:35 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735114/ [5735115] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-snaps} time: 00:21:41 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735115/ [5735116] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/cosbench_64K_read_write} time: 00:19:03 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735116/ [5735117] rados/cephadm/smoke-roleless/{distro/rhel_8.0 start} time: 00:14:21 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735117/ [5735118] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:18:17 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735118/ [5735119] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:18:18 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735119/ [5735120] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/none thrashosds-health workloads/radosbench} time: 01:08:18 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735120/ [5735121] rados/standalone/{mon_election/connectivity supported-random-distro$/{ubuntu_latest} workloads/misc} time: 00:32:49 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735121/ [5735122] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/connectivity start} time: 00:13:25 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735122/ [5735123] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8} tasks/dashboard} time: 01:12:06 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735123/ [5735124] rados/objectstore/{backends/filejournal supported-random-distro$/{rhel_8}} time: 00:09:07 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735124/ [5735125] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache} time: 00:11:06 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735125/ [5735126] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_mix} time: 00:28:47 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735126/ [5735127] rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:29:19 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735127/ [5735128] rados/singleton/{all/watch-notify-same-primary mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:09:16 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735128/ [5735129] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:13:34 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735129/ [5735130] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zstd supported-random-distro$/{rhel_8} tasks/workunits} time: 00:16:41 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735130/ [5735131] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/classic start} time: 00:13:05 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735131/ [5735132] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:44:33 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735132/ [5735133] rados/cephadm/smoke-roleless/{distro/rhel_latest start} time: 00:11:48 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735133/ [5735135] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:09:04 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735135/ [5735136] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:19:18 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735136/ [5735137] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/cosbench_64K_write} time: 00:26:20 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735137/ [5735138] rados/cephadm/workunits/{distro/centos_latest mon_election/classic task/test_cephadm} time: 00:21:07 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735138/ [5735139] rados/singleton/{all/admin-socket mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:09:18 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735139/ [5735140] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:09:57 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735140/ [5735141] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/rbd_cls} time: 00:20:46 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735141/ [5735142] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:12:01 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735142/ [5735143] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados tasks/mon_recovery validater/lockdep} time: 00:12:37 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735143/ [5735144] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-hybrid supported-random-distro$/{centos_8} tasks/dashboard} time: 01:09:24 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735144/ [5735145] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:26 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735145/ [5735146] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/connectivity start} time: 00:14:09 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735146/ [5735148] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/rados_api_tests} time: 00:37:00 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735148/ [5735149] rados/standalone/{mon_election/classic supported-random-distro$/{ubuntu_latest} workloads/mon} time: 00:45:38 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735149/ [5735150] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/sync workloads/pool-create-delete} time: 00:14:54 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735150/ [5735151] rados/singleton/{all/deduptool mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:13:33 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735151/ [5735153] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:18:03 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735153/ [5735154] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/careful thrashosds-health workloads/snaps-few-objects} time: 00:36:38 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735154/ [5735155] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mostlyread} time: 00:40:02 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735155/ [5735156] rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:38:57 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735156/ [5735157] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/radosbench-high-concurrency} time: 00:48:26 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735157/ [5735158] rados/singleton/{all/divergent_priors mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:07:12 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735158/ [5735159] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:10:40 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735159/ [5735160] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/classic start} time: 00:13:28 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735160/ [5735161] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:15:09 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735161/ [5735162] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/fastclose rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:24:19 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735162/ [5735163] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-hybrid supported-random-distro$/{centos_8} tasks/crash} time: 00:11:21 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735163/ [5735164] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04 start} time: 00:11:37 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735164/ [5735165] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:08:00 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735165/ [5735166] rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{rhel_8}} time: 02:15:31 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735166/ [5735167] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:19:50 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735167/ [5735168] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/connectivity task/test_cephadm_repos} time: 00:11:27 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735168/ [5735169] rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:07:18 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735169/ [5735170] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/radosbench} time: 02:23:58 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735170/ [5735171] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:18:49 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735171/ [5735172] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/connectivity start} time: 00:15:09 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735172/ [5735173] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:13:13 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735173/ [5735174] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-small-objects} time: 00:22:40 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735174/ [5735175] rados/standalone/{mon_election/connectivity supported-random-distro$/{centos_8} workloads/osd} time: 03:12:15 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735175/ [5735176] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/default thrashosds-health workloads/test_rbd_api} time: 00:18:51 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735176/ [5735177] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:40 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735177/ [5735178] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:10:03 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735178/ [5735179] rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_python} time: 00:28:55 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735179/ [5735182] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/redirect} time: 00:10:27 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735182/ [5735183] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:16:13 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735183/ [5735184] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:11:07 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735184/ [5735185] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:24:44 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735185/ [5735186] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/classic start} time: 00:12:33 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735186/ [5735187] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/rados_5925} time: 00:09:09 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735187/ [5735188] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-stupid supported-random-distro$/{centos_8} tasks/dashboard} time: 01:10:55 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735188/ [5735189] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} tasks/readwrite} time: 00:11:59 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735189/ [5735190] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman start} time: 00:20:59 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735190/ [5735191] rados/singleton/{all/ec-lost-unfound mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:39:27 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735191/ [5735193] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/redirect_promote_tests} time: 00:13:36 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735193/ [5735194] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:16:13 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735194/ [5735196] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} time: 00:12:34 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735196/ [5735197] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} tasks/failover} time: 00:11:03 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735197/ [5735198] rados/singleton/{all/erasure-code-nonregression mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:14:45 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735198/ [5735199] rados/standalone/{mon_election/classic supported-random-distro$/{centos_8} workloads/scrub} time: 01:46:01 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735199/ [5735200] rados/cephadm/workunits/{distro/centos_latest mon_election/classic task/test_orch_cli} time: 00:19:28 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735200/ [5735201] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/redirect_set_object} time: 00:13:38 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735201/ [5735202] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:28:25 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735202/ [5735203] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/filestore-xfs supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:08:32 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735203/ [5735204] rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{centos_8}} time: 02:25:27 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735204/ [5735205] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:10:31 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735205/ [5735206] rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:40:18 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735206/ [5735207] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:09:08 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735207/ [5735208] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:28:25 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735208/ [5735209] rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:29:51 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735209/ [5735210] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/classic start} time: 00:13:42 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735210/ [5735211] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:54 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735211/ [5735212] rados/cephadm/smoke-roleless/{distro/ubuntu_latest start} time: 00:11:48 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735212/ [5735213] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:17:21 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735213/ [5735214] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/set-chunks-read} time: 00:17:23 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735214/ [5735215] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:28:27 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735215/ [5735216] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:11:52 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735216/ [5735217] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados tasks/rados_cls_all validater/lockdep} time: 00:25:28 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735217/ [5735218] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/radosbench} time: 01:18:47 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735218/ [5735219] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} tasks/repair_test} time: 00:16:51 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735219/ [5735220] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:23:37 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735220/ [5735221] rados/singleton/{all/lost-unfound mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:49:12 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735221/ [5735222] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/connectivity start} time: 00:12:36 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735222/ [5735223] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/many workloads/rados_api_tests} time: 00:38:13 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735223/ [5735224] rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:29:13 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735224/ [5735225] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:45:42 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735225/ [5735226] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:08:24 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735226/ [5735227] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:15:40 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735227/ [5735228] rados/standalone/{mon_election/connectivity supported-random-distro$/{rhel_8} workloads/crush} time: 00:18:30 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735228/ [5735229] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-octopus 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/connectivity thrashosds-health ubuntu_latest} time: 05:08:09 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735229/ [5735230] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/small-objects-balanced} time: 00:18:12 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735230/ [5735231] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:09:12 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735231/ [5735232] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/none thrashosds-health workloads/rbd_cls} time: 00:15:28 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735232/ [5735233] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:05:45 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735233/ [5735234] rados/cephadm/workunits/{distro/centos_latest mon_election/classic task/test_adoption} time: 00:15:51 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735234/ [5735235] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-stupid supported-random-distro$/{centos_8} tasks/insights} time: 00:13:20 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735235/ [5735236] rados/cephadm/dashboard/{distro/centos_latest task/test_e2e} time: 00:31:53 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735236/ [5735237] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:14:30 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735237/ [5735238] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/small-objects-localized} time: 00:23:48 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735238/ [5735239] rados/cephadm/orchestrator_cli/{2-node-mgr orchestrator_cli supported-random-distro$/{rhel_8}} time: 00:13:29 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735239/ [5735240] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:09:48 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735240/ [5735241] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/classic start} time: 00:14:22 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735241/ [5735242] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:07:59 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735242/ [5735243] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:24:46 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735243/ [5735244] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:08:01 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735244/ [5735245] rados/objectstore/{backends/fusestore supported-random-distro$/{rhel_8}} time: 00:15:23 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735245/ [5735246] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:59 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735246/ [5735247] rados/cephadm/smoke-roleless/{distro/centos_8.0 start} time: 00:12:56 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735247/ [5735248] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:09:41 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735248/ [5735249] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/scrub_test} time: 00:09:33 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735249/ [5735250] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/small-objects} time: 00:24:55 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735250/ [5735251] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:12:39 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735251/ [5735253] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:09:08 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735253/ [5735254] rados/standalone/{mon_election/classic supported-random-distro$/{ubuntu_latest} workloads/erasure-code} time: 00:41:27 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735254/ [5735255] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:26:37 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735255/ [5735256] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados tasks/mon_recovery validater/valgrind} time: 00:28:07 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735256/ [5735257] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/connectivity start} time: 00:13:16 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735257/ [5735258] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:12 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735258/ [5735259] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/one workloads/rados_mon_osdmap_prune} time: 01:08:38 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735259/ [5735260] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:16:48 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735260/ [5735261] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/snaps-few-objects} time: 00:36:24 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735261/ [5735262] rados/singleton/{all/mon-auth-caps mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:17:21 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735262/ [5735263] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/snaps-few-objects-balanced} time: 00:26:59 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735263/ [5735264] rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:42:24 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735264/ [5735265] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/classic start} time: 00:15:25 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735265/ [5735266] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8} tasks/dashboard} time: 01:08:59 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735266/ [5735267] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:24:10 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735267/ [5735268] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/filestore-xfs supported-random-distro$/{centos_8} tasks/module_selftest} time: 00:27:55 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735268/ [5735269] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:14:49 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735269/ [5735270] rados/cephadm/smoke-roleless/{distro/centos_latest start} time: 00:12:27 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735270/ [5735271] rados/singleton/{all/mon-config-key-caps mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:18:48 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735271/ [5735274] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:11:27 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735274/ [5735275] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/careful thrashosds-health workloads/test_rbd_api} time: 00:18:55 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735275/ [5735276] rados/standalone/{mon_election/connectivity supported-random-distro$/{ubuntu_latest} workloads/mgr} time: 00:15:53 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735276/ [5735278] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:05:43 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735278/ [5735279] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:23 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735279/ [5735280] rados/singleton/{all/mon-config-keys mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:21:03 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735280/ [5735281] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/connectivity start} time: 00:12:40 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735281/ [5735282] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:12:59 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735282/ [5735283] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:24:47 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735283/ [5735284] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} time: 00:29:53 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735284/ [5735286] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zlib supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:08:37 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735286/ [5735287] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{rhel_8}} time: 00:13:47 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735287/ [5735288] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:18:58 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735288/ [5735289] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:11:14 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735289/ [5735290] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/filestore-xfs rados tasks/rados_api_tests validater/lockdep} time: 00:32:51 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735290/ [5735291] rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_python} time: 00:29:45 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735291/ [5735292] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:08:39 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735292/ [5735293] rados/singleton/{all/mon-config mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:16:40 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735293/ [5735294] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:23:14 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735294/ [5735295] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:12:53 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735295/ [5735296] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/sync-many workloads/rados_mon_workunits} time: 00:33:47 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735296/ [5735297] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/write_fadvise_dontneed} time: 00:10:05 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735297/ [5735298] rados/cephadm/smoke-roleless/{distro/rhel_8.0 start} time: 00:14:18 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735298/ [5735299] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:09:47 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735299/ [5735300] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/default thrashosds-health workloads/cache-snaps} time: 00:30:01 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735300/ [5735301] rados/singleton/{all/osd-backfill mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:14:05 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735301/ [5735302] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:17:39 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735302/ [5735303] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/progress} time: 00:22:39 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735303/ [5735304] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/connectivity start} time: 00:13:07 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735304/ [5735305] rados/standalone/{mon_election/classic supported-random-distro$/{rhel_8} workloads/misc} time: 00:34:54 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735305/ [5735306] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/admin_socket_objecter_requests} time: 00:26:37 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735306/ [5735307] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:06:17 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735307/ [5735308] rados/cephadm/workunits/{distro/centos_latest mon_election/classic task/test_cephadm_repos} time: 00:13:07 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735308/ [5735309] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/rados_cls_all} time: 00:23:25 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735309/ [5735310] rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:38:53 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735310/ [5735311] rados/singleton/{all/osd-recovery-incomplete mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:17:52 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735311/ [5735312] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:13:18 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735312/ [5735313] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:30 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735313/ [5735314] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/mapgap thrashosds-health workloads/radosbench} time: 01:11:34 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735314/ [5735316] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache-agent-big} time: 00:30:32 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735316/ [5735317] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:08:19 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735317/ [5735318] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-radosbench} time: 02:16:05 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735318/ [5735319] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/classic start} time: 00:12:51 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735319/ [5735320] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:10:36 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735320/ [5735321] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:25:23 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735321/ [5735322] rados/singleton/{all/osd-recovery mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:11:44 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735322/ [5735323] rados/cephadm/smoke-roleless/{distro/rhel_latest start} time: 00:11:40 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735323/ [5735324] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:07:00 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735324/ [5735325] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:27:22 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735325/ [5735327] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/connectivity start} time: 00:14:02 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735327/ [5735328] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-hybrid supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:08:00 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735328/ [5735329] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{rhel_8}} time: 00:30:55 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735329/ [5735330] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:24 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735330/ [5735331] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/cache-agent-small} time: 00:18:22 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735331/ [5735332] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{ubuntu_18.04_podman} fixed-2 mon_election/classic} time: 00:19:45 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735332/ [5735333] rados/standalone/{mon_election/connectivity supported-random-distro$/{centos_8} workloads/mon} time: 00:52:53 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735333/ [5735334] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/sync workloads/snaps-few-objects} time: 00:28:34 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735334/ [5735335] rados/singleton/{all/peer mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:06:03 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735335/ [5735336] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:13:58 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735336/ [5735337] rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:29:11 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735337/ [5735338] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/rbd_cls} time: 00:19:28 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735338/ [5735339] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8} tasks/prometheus} time: 00:11:55 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735339/ [5735340] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} tasks/rados_python} time: 00:22:26 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735340/ [5735341] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:22:35 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735341/ [5735342] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/connectivity task/test_orch_cli} time: 00:17:10 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735342/ [5735343] rados/singleton/{all/pg-autoscaler-progress-off mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:15:34 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735343/ [5735344] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:08:42 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735344/ [5735345] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:13:54 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735345/ [5735346] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/classic start} time: 00:13:06 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735346/ [5735347] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:07:07 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735347/ [5735348] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:15 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735348/ [5735349] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04 start} time: 00:11:30 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735349/ [5735352] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/connectivity start} time: 00:15:31 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735352/ [5735353] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:08:54 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735353/ [5735354] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-balanced} time: 00:22:11 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735354/ [5735355] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:21:58 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735355/ [5735356] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/none thrashosds-health workloads/snaps-few-objects} time: 00:30:51 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735356/ [5735357] rados/standalone/{mon_election/classic supported-random-distro$/{ubuntu_latest} workloads/osd} time: 03:15:46 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735357/ [5735358] rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:40:16 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735358/ [5735359] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:09:20 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735359/ [5735360] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep} time: 00:25:39 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735360/ [5735361] rados/singleton/{all/pg-removal-interruption mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:09:18 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735361/ [5735362] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-snaps-balanced} time: 00:25:02 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735362/ [5735364] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/classic start} time: 00:12:37 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735364/ [5735365] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:08:23 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735365/ [5735366] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:16:06 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735366/ [5735367] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/pool-create-delete} time: 00:14:30 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735367/ [5735368] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman start} time: 00:12:29 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735368/ [5735369] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-stupid supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:09:48 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735369/ [5735370] rados/objectstore/{backends/objectstore supported-random-distro$/{ubuntu_latest}} time: 05:16:37 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735370/ [5735371] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/rados_stress_watch} time: 00:24:23 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735371/ [5735372] rados/cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_adoption} time: 00:15:39 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735372/ [5735373] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/few rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:20:33 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735373/ [5735374] rados/singleton/{all/radostool mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:19:13 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735374/ [5735375] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8} tasks/workunits} time: 00:16:08 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735375/ [5735376] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache-snaps} time: 00:17:12 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735376/ [5735377] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{ubuntu_18.04} fixed-2 mon_election/connectivity} time: 00:17:49 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735377/ [5735378] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:16:05 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735378/ [5735379] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/test_rbd_api} time: 00:18:43 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735379/ [5735380] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:44 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735380/ [5735381] rados/singleton/{all/random-eio mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:30:20 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735381/ [5735382] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} time: 00:12:49 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735382/ [5735383] rados/standalone/{mon_election/connectivity supported-random-distro$/{ubuntu_latest} workloads/scrub} time: 01:45:00 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735383/ [5735384] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache} time: 00:14:02 info: http://pulpito.ceph.com/teuthology-2020-12-24_07:01:02-rados-master-distro-basic-smithi/5735384/