2020-11-25T20:50:15.809 INFO:root:teuthology version: 1.0.0-e0aaa9bf 2020-11-25T20:50:15.811 INFO:teuthology.results:subset: '23/100000' 2020-11-25T20:50:15.811 INFO:teuthology.results:seed: '1359' 2020-11-25T23:36:02.580 INFO:root:teuthology version: 1.0.0-e0aaa9bf 2020-11-25T23:36:02.602 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-11-26T04:49:14.688 INFO:teuthology.results:Tests finished! gathering results... 2020-11-26T04:50:05.048 INFO:scrape:Found 367 jobs 2020-11-26T04:50:15.326 INFO:scrape:Found 9 distinct failure reasons 2020-11-26T04:50:15.327 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi061 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4ee600b3548e2e5c35f5d3130ad1de25181d67ff TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2020-11-26T04:50:15.327 INFO:scrape:2 jobs: ['5653079', '5653259'] 2020-11-26T04:50:15.328 INFO:scrape:suites intersection: ['rados/cephadm/dashboard/{distro/centos_latest', 'task/test_e2e}'] 2020-11-26T04:50:15.328 INFO:scrape:suites union: ['rados/cephadm/dashboard/{distro/centos_latest', 'task/test_e2e}'] 2020-11-26T04:50:15.329 INFO:scrape: 2020-11-26T04:50:15.329 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/mon/caps.sh 2020-11-26T04:50:15.329 INFO:scrape:2 jobs: ['5653362', '5653104'] 2020-11-26T04:50:15.330 INFO:scrape:suites intersection: ['rados', 'rados/monthrash/{ceph', 'supported-random-distro$/{centos_8}', 'workloads/rados_mon_workunits}'] 2020-11-26T04:50:15.330 INFO:scrape:suites union: ['clusters/3-mons', 'clusters/9-mons', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/few', 'msgr-failures/mon-delay', 'msgr/async', 'msgr/async-v2only', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-stupid', 'rados', 'rados/monthrash/{ceph', 'supported-random-distro$/{centos_8}', 'thrashers/one', 'thrashers/sync', 'workloads/rados_mon_workunits}'] 2020-11-26T04:50:15.331 INFO:scrape: 2020-11-26T04:50:15.331 INFO:scrape:Failure: reached maximum tries (180) after waiting for 180 seconds 2020-11-26T04:50:15.331 INFO:scrape:4 jobs: ['5653220', '5653314', '5653357', '5653402'] 2020-11-26T04:50:15.332 INFO:scrape:suites intersection: ['3-start-upgrade', '4-wait', 'fixed-2', 'rados/cephadm/upgrade/{1-start'] 2020-11-26T04:50:15.332 INFO:scrape:suites union: ['2-repo_digest/defaut', '2-repo_digest/repo_digest', '3-start-upgrade', '4-wait', 'distro$/{rhel_8.0}', 'distro$/{rhel_latest}', 'fixed-2', 'mon_election/classic}', 'mon_election/connectivity}', 'rados/cephadm/upgrade/{1-start'] 2020-11-26T04:50:15.333 INFO:scrape: 2020-11-26T04:50:15.333 INFO:scrape:Failure: "2020-11-25T23:12:31.830752+0000 mon.b (mon.0) 11 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,a (MON_DOWN)" in cluster log 2020-11-26T04:50:15.333 INFO:scrape:1 jobs: ['5653326'] 2020-11-26T04:50:15.334 INFO:scrape:suites: ['ceph', 'clusters/{fixed-2', 'mon_election/connectivity', 'msgr-failures/osd-delay', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/thrash-erasure-code-isa/{arch/x86_64', 'recovery-overrides/{more-active-recovery}', 'supported-random-distro$/{rhel_8}', 'thrashers/pggrow', 'thrashosds-health', 'workloads/ec-rados-plugin=isa-k=2-m=1}'] 2020-11-26T04:50:15.334 INFO:scrape: 2020-11-26T04:50:15.335 INFO:scrape:Failure: Command failed (workunit test cephadm/test_cephadm.sh) on smithi105 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4ee600b3548e2e5c35f5d3130ad1de25181d67ff TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' 2020-11-26T04:50:15.335 INFO:scrape:2 jobs: ['5653204', '5653343'] 2020-11-26T04:50:15.335 INFO:scrape:suites intersection: ['rados/cephadm/workunits/{distro/centos_latest', 'task/test_cephadm}'] 2020-11-26T04:50:15.336 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{distro/centos_latest', 'task/test_cephadm}'] 2020-11-26T04:50:15.336 INFO:scrape: 2020-11-26T04:50:15.337 INFO:scrape:Failure: Command failed on smithi039 with status 5: 'sudo systemctl stop ceph-d2940766-2f64-11eb-980d-001a4aab830c@mon.a' 2020-11-26T04:50:15.337 INFO:scrape:2 jobs: ['5653092', '5653126'] 2020-11-26T04:50:15.337 INFO:scrape:suites intersection: ['3-start-upgrade', '4-wait', 'distro$/{ubuntu_18.04}', 'fixed-2', 'rados/cephadm/upgrade/{1-start'] 2020-11-26T04:50:15.338 INFO:scrape:suites union: ['2-repo_digest/defaut', '2-repo_digest/repo_digest', '3-start-upgrade', '4-wait', 'distro$/{ubuntu_18.04}', 'fixed-2', 'mon_election/classic}', 'mon_election/connectivity}', 'rados/cephadm/upgrade/{1-start'] 2020-11-26T04:50:15.338 INFO:scrape: 2020-11-26T04:50:15.339 INFO:scrape:Failure: Command failed (workunit test cephadm/test_adoption.sh) on smithi012 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4ee600b3548e2e5c35f5d3130ad1de25181d67ff TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_adoption.sh' 2020-11-26T04:50:15.339 INFO:scrape:2 jobs: ['5653168', '5653308'] 2020-11-26T04:50:15.339 INFO:scrape:suites intersection: ['rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_adoption}'] 2020-11-26T04:50:15.340 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_adoption}'] 2020-11-26T04:50:15.340 INFO:scrape: 2020-11-26T04:50:15.341 INFO:scrape:Failure: Command failed (workunit test rados/test.sh) on smithi045 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4ee600b3548e2e5c35f5d3130ad1de25181d67ff TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' 2020-11-26T04:50:15.341 INFO:scrape:1 jobs: ['5653367'] 2020-11-26T04:50:15.341 INFO:scrape:suites: ['clusters/{fixed-2', 'mon_election/classic', 'msgr-failures/few', 'msgr/async', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/basic/{ceph', 'supported-random-distro$/{centos_8}', 'tasks/rados_api_tests}'] 2020-11-26T04:50:15.342 INFO:scrape: 2020-11-26T04:50:15.342 INFO:scrape:Failure: Test failure: test_cluster_info (tasks.cephfs.test_nfs.TestNFS) 2020-11-26T04:50:15.343 INFO:scrape:1 jobs: ['5653145'] 2020-11-26T04:50:15.343 INFO:scrape:suites: ['mon_election/classic', 'rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_orch_cli}'] 2020-11-26T04:50:15.343 INFO:scrape: 2020-11-26T04:50:15.355 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/ failed: 17 dead: 0 running: 0 waiting: 0 queued: 0 passed: 350 Fail ================================================================= [5653079] rados/cephadm/dashboard/{distro/centos_latest task/test_e2e} ----------------------------------------------------------------- time: 00:31:05 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653079/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653079/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a890494ff7464a7a9b5770956e044a63 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi061 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4ee600b3548e2e5c35f5d3130ad1de25181d67ff TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [5653092] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{ubuntu_18.04} fixed-2 mon_election/classic} ----------------------------------------------------------------- time: 00:02:10 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653092/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653092/ sentry: https://sentry.ceph.com/organizations/ceph/?query=47cb232aff2840459133f1cdf4b5bb58 Command failed on smithi039 with status 5: 'sudo systemctl stop ceph-d2940766-2f64-11eb-980d-001a4aab830c@mon.a' [5653104] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/one workloads/rados_mon_workunits} ----------------------------------------------------------------- time: 03:19:47 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653104/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653104/ sentry: https://sentry.ceph.com/organizations/ceph/?query=38ff6237760b4e128b4810e8230dda19 Command failed (workunit test mon/caps.sh) on smithi042 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4ee600b3548e2e5c35f5d3130ad1de25181d67ff TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/caps.sh' [5653126] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{ubuntu_18.04} fixed-2 mon_election/connectivity} ----------------------------------------------------------------- time: 00:02:10 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653126/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653126/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f5c8948b8dcc4e039ed41ff2433efb2f Command failed on smithi038 with status 5: 'sudo systemctl stop ceph-75c903d0-2f67-11eb-980d-001a4aab830c@mon.a' [5653145] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/classic task/test_orch_cli} ----------------------------------------------------------------- time: 00:10:56 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653145/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653145/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fba5d7dc176a49ef8e89cb55bb88d394 Test failure: test_cluster_info (tasks.cephfs.test_nfs.TestNFS) [5653168] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/classic task/test_adoption} ----------------------------------------------------------------- time: 00:11:03 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653168/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653168/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0b6fd47b5db34526b6cf409f47d7ab6a Command failed (workunit test cephadm/test_adoption.sh) on smithi012 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4ee600b3548e2e5c35f5d3130ad1de25181d67ff TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_adoption.sh' [5653204] rados/cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:10:44 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653204/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653204/ sentry: https://sentry.ceph.com/organizations/ceph/?query=27c0066b33d04c88a7d485e550f946e2 Command failed (workunit test cephadm/test_cephadm.sh) on smithi105 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4ee600b3548e2e5c35f5d3130ad1de25181d67ff TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [5653220] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{rhel_latest} fixed-2 mon_election/connectivity} ----------------------------------------------------------------- time: 00:14:43 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653220/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653220/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4efec38febb84b44ac7d4581c90459fd reached maximum tries (180) after waiting for 180 seconds [5653259] rados/cephadm/dashboard/{distro/centos_latest task/test_e2e} ----------------------------------------------------------------- time: 00:28:15 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653259/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653259/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f420f5cf42a1457c88677d68f9281aa7 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi023 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4ee600b3548e2e5c35f5d3130ad1de25181d67ff TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [5653308] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/connectivity task/test_adoption} ----------------------------------------------------------------- time: 00:11:14 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653308/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653308/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4276a572471a4526899a5db9810f2660 Command failed (workunit test cephadm/test_adoption.sh) on smithi152 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4ee600b3548e2e5c35f5d3130ad1de25181d67ff TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_adoption.sh' [5653314] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{rhel_8.0} fixed-2 mon_election/connectivity} ----------------------------------------------------------------- time: 00:18:11 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653314/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653314/ sentry: https://sentry.ceph.com/organizations/ceph/?query=837cb523bbb24c9c973c36a82e7795f7 reached maximum tries (180) after waiting for 180 seconds [5653326] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:29:33 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653326/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653326/ "2020-11-25T23:12:31.830752+0000 mon.b (mon.0) 11 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,a (MON_DOWN)" in cluster log [5653343] rados/cephadm/workunits/{distro/centos_latest mon_election/classic task/test_cephadm} ----------------------------------------------------------------- time: 00:10:06 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653343/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653343/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d16287544c2b4f118b50cd3b946baec4 Command failed (workunit test cephadm/test_cephadm.sh) on smithi190 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4ee600b3548e2e5c35f5d3130ad1de25181d67ff TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [5653357] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{rhel_8.0} fixed-2 mon_election/classic} ----------------------------------------------------------------- time: 00:19:09 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653357/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653357/ sentry: https://sentry.ceph.com/organizations/ceph/?query=48d8629337d248e195e1b87b215887d5 reached maximum tries (180) after waiting for 180 seconds [5653362] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/sync workloads/rados_mon_workunits} ----------------------------------------------------------------- time: 03:21:11 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653362/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653362/ sentry: https://sentry.ceph.com/organizations/ceph/?query=88bea00ed76d471d8d190440f2cc66d4 Command failed (workunit test mon/caps.sh) on smithi200 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4ee600b3548e2e5c35f5d3130ad1de25181d67ff TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/caps.sh' [5653367] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:24:17 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653367/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653367/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a45dd8bc9e8f4cf8ad1afbf88cd1c6de Command failed (workunit test rados/test.sh) on smithi045 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4ee600b3548e2e5c35f5d3130ad1de25181d67ff TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [5653402] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{rhel_8.0} fixed-2 mon_election/connectivity} ----------------------------------------------------------------- time: 00:18:25 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653402/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653402/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1f7a0db232854b0dbd20ff7210d2cc25 reached maximum tries (180) after waiting for 180 seconds Pass ================================================================= [5653044] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:27:46 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653044/ [5653045] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/filestore-xfs supported-random-distro$/{centos_8} tasks/dashboard} time: 01:10:28 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653045/ [5653046] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/classic start} time: 00:13:11 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653046/ [5653047] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-stupid supported-random-distro$/{centos_8} tasks/module_selftest} time: 00:29:01 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653047/ [5653048] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:28:35 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653048/ [5653049] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:09:21 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653049/ [5653050] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:06:26 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653050/ [5653051] rados/cephadm/smoke-roleless/{distro/ubuntu_latest start} time: 00:10:23 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653051/ [5653052] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:48 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653052/ [5653053] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:07:50 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653053/ [5653054] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-small-objects} time: 00:20:51 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653054/ [5653055] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:12:26 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653055/ [5653056] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/cache-snaps} time: 00:29:37 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653056/ [5653057] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/rados_cls_all} time: 00:20:38 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653057/ [5653058] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/connectivity start} time: 00:12:02 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653058/ [5653059] rados/singleton/{all/radostool mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:15:00 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653059/ [5653060] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-pool-snaps} time: 00:24:23 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653060/ [5653061] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/classic task/test_cephadm} time: 00:18:20 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653061/ [5653062] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:37:28 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653062/ [5653063] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/many workloads/rados_mon_osdmap_prune} time: 01:03:26 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653063/ [5653064] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/dashboard} time: 01:09:32 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653064/ [5653065] rados/objectstore/{backends/alloc-hint supported-random-distro$/{centos_8}} time: 00:10:44 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653065/ [5653066] rados/rest/{mgr-restful supported-random-distro$/{rhel_8}} time: 00:12:47 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653066/ [5653067] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:12:56 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653067/ [5653068] rados/standalone/{mon_election/classic supported-random-distro$/{rhel_8} workloads/crush} time: 00:14:37 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653068/ [5653069] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-octopus 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/classic thrashosds-health ubuntu_latest} time: 04:51:45 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653069/ [5653070] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:17:52 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653070/ [5653071] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/careful thrashosds-health workloads/radosbench} time: 01:31:31 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653071/ [5653072] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:24:18 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653072/ [5653073] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados tasks/rados_api_tests validater/valgrind} time: 00:49:19 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653073/ [5653074] rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:25:52 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653074/ [5653075] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:16 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653075/ [5653076] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:22:01 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653076/ [5653077] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-snaps-balanced} time: 00:22:22 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653077/ [5653078] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:09:16 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653078/ [5653080] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:12:06 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653080/ [5653081] rados/cephadm/orchestrator_cli/{2-node-mgr orchestrator_cli supported-random-distro$/{centos_8}} time: 00:11:35 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653081/ [5653082] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:16:01 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653082/ [5653083] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/classic start} time: 00:14:21 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653083/ [5653084] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/filestore-xfs supported-random-distro$/{ubuntu_latest} tasks/progress} time: 00:11:26 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653084/ [5653085] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:27:51 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653085/ [5653086] rados/cephadm/smoke-roleless/{distro/centos_8.0 start} time: 00:12:47 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653086/ [5653087] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/fastclose rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:22:28 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653087/ [5653088] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:13:02 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653088/ [5653089] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:08 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653089/ [5653090] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:06:45 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653090/ [5653091] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/rados_python} time: 00:15:49 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653091/ [5653093] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:15:38 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653093/ [5653094] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:07:54 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653094/ [5653095] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:30:19 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653095/ [5653096] rados/standalone/{mon_election/connectivity supported-random-distro$/{rhel_8} workloads/erasure-code} time: 00:40:41 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653096/ [5653097] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/connectivity start} time: 00:11:20 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653097/ [5653098] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache} time: 00:10:42 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653098/ [5653099] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:08:00 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653099/ [5653100] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/default thrashosds-health workloads/rbd_cls} time: 00:19:54 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653100/ [5653101] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:11:42 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653101/ [5653102] rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} time: 00:40:07 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653102/ [5653103] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:06:20 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653103/ [5653105] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:08:51 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653105/ [5653106] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/lockdep} time: 00:31:33 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653106/ [5653107] rados/cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_cephadm_repos} time: 00:07:59 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653107/ [5653108] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/pool-snaps-few-objects} time: 00:26:50 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653108/ [5653109] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8} tasks/dashboard} time: 01:10:28 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653109/ [5653110] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{rhel_8}} time: 00:11:38 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653110/ [5653111] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:33 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653111/ [5653112] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/classic start} time: 00:15:29 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653112/ [5653113] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:09:27 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653113/ [5653114] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:14:08 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653114/ [5653115] rados/cephadm/smoke-roleless/{distro/centos_latest start} time: 00:10:07 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653115/ [5653116] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:36:34 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653116/ [5653117] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/prometheus} time: 00:12:25 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653117/ [5653118] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/rados_api_tests} time: 00:26:32 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653118/ [5653119] rados/standalone/{mon_election/classic supported-random-distro$/{centos_8} workloads/mgr} time: 00:13:32 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653119/ [5653120] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:22:38 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653120/ [5653121] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/connectivity start} time: 00:11:58 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653121/ [5653122] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} tasks/rados_stress_watch} time: 00:18:59 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653122/ [5653123] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:07:50 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653123/ [5653124] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:36 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653124/ [5653125] rados/singleton/{all/test_envlibrados_for_rocksdb mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:19:11 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653125/ [5653127] rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_python} time: 00:25:20 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653127/ [5653128] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zlib supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:10:58 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653128/ [5653129] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:09:28 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653129/ [5653130] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:25:08 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653130/ [5653131] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/radosbench-high-concurrency} time: 00:30:39 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653131/ [5653132] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:12:23 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653132/ [5653133] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:05:20 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653133/ [5653134] rados/singleton/{all/thrash-backfill-full mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 01:03:17 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653134/ [5653135] rados/cephadm/smoke-roleless/{distro/rhel_8.0 start} time: 00:14:13 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653135/ [5653136] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-small-objects-overwrites} time: 00:22:19 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653136/ [5653137] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/test_rbd_api} time: 00:18:38 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653137/ [5653138] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/snaps-few-objects} time: 00:27:03 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653138/ [5653139] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:30:25 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653139/ [5653140] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/valgrind} time: 02:07:48 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653140/ [5653141] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/radosbench} time: 02:20:25 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653141/ [5653142] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_fio} time: 00:11:20 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653142/ [5653143] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:13:26 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653143/ [5653144] rados/singleton/{all/thrash-eio mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:38:45 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653144/ [5653146] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:20 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653146/ [5653147] rados/standalone/{mon_election/connectivity supported-random-distro$/{rhel_8} workloads/misc} time: 00:30:44 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653147/ [5653148] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/connectivity start} time: 00:13:13 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653148/ [5653149] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8} tasks/dashboard} time: 01:11:23 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653149/ [5653150] rados/objectstore/{backends/filejournal supported-random-distro$/{ubuntu_latest}} time: 00:05:13 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653150/ [5653151] rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} time: 00:34:35 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653151/ [5653152] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/rados_striper} time: 00:06:46 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653152/ [5653153] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:24:18 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653153/ [5653154] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/redirect} time: 00:11:38 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653154/ [5653155] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:12:19 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653155/ [5653156] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} tasks/workunits} time: 00:10:15 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653156/ [5653157] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/none thrashosds-health workloads/cache-snaps} time: 00:25:13 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653157/ [5653158] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:53 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653158/ [5653159] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:44:13 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653159/ [5653160] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/classic start} time: 00:12:38 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653160/ [5653161] rados/cephadm/smoke-roleless/{distro/rhel_latest start} time: 00:10:45 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653161/ [5653162] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:34:42 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653162/ [5653163] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:08:46 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653163/ [5653164] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/redirect_promote_tests} time: 00:10:52 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653164/ [5653165] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:08:19 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653165/ [5653166] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:04:55 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653166/ [5653167] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:22:57 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653167/ [5653169] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-hybrid supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:12:10 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653169/ [5653170] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/connectivity start} time: 00:15:10 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653170/ [5653171] rados/standalone/{mon_election/classic supported-random-distro$/{rhel_8} workloads/mon} time: 00:52:51 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653171/ [5653172] rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade 4-wait distro$/{centos_latest} fixed-2 mon_election/classic} time: 00:14:23 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653172/ [5653173] rados/singleton/{all/watch-notify-same-primary mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:08:13 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653173/ [5653174] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:10:29 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653174/ [5653175] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/redirect_set_object} time: 00:12:52 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653175/ [5653176] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/sync workloads/snaps-few-objects} time: 00:27:02 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653176/ [5653177] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/pggrow thrashosds-health workloads/radosbench} time: 01:35:13 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653177/ [5653178] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:08:34 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653178/ [5653179] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/lockdep} time: 00:09:49 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653179/ [5653180] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:32:17 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653180/ [5653181] rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_python} time: 00:26:02 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653181/ [5653182] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_big} time: 00:35:23 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653182/ [5653183] rados/singleton/{all/admin-socket mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:07:35 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653183/ [5653184] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/classic start} time: 00:11:38 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653184/ [5653185] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:14:15 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653185/ [5653186] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/set-chunks-read} time: 00:13:00 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653186/ [5653187] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/cosbench_64K_read_write} time: 00:20:47 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653187/ [5653188] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04 start} time: 00:10:37 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653188/ [5653189] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:08:48 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653189/ [5653190] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8} tasks/dashboard} time: 01:12:07 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653190/ [5653191] rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{rhel_8}} time: 02:22:13 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653191/ [5653192] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:09:50 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653192/ [5653193] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:17:04 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653193/ [5653194] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:26:38 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653194/ [5653195] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/careful thrashosds-health workloads/rbd_cls} time: 00:19:09 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653195/ [5653196] rados/singleton/{all/deduptool mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:10:47 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653196/ [5653197] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/connectivity start} time: 00:15:36 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653197/ [5653198] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:22:35 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653198/ [5653199] rados/standalone/{mon_election/connectivity supported-random-distro$/{centos_8} workloads/osd} time: 03:08:03 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653199/ [5653200] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/small-objects-balanced} time: 00:18:59 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653200/ [5653201] rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} time: 00:36:58 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653201/ [5653202] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:07:26 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653202/ [5653203] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-radosbench} time: 01:19:43 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653203/ [5653205] rados/singleton/{all/divergent_priors mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:06:31 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653205/ [5653206] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/classic start} time: 00:11:50 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653206/ [5653207] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:15:34 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653207/ [5653208] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-stupid supported-random-distro$/{centos_8} tasks/dashboard} time: 01:07:39 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653208/ [5653209] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/small-objects-localized} time: 00:21:55 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653209/ [5653210] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/cosbench_64K_write} time: 00:12:39 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653210/ [5653211] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman start} time: 00:11:08 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653211/ [5653212] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_mix} time: 00:23:18 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653212/ [5653213] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/pool-create-delete} time: 00:16:32 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653213/ [5653214] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:27:24 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653214/ [5653215] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/valgrind} time: 00:48:26 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653215/ [5653216] rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:08:05 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653216/ [5653217] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/default thrashosds-health workloads/snaps-few-objects} time: 00:35:11 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653217/ [5653218] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:26 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653218/ [5653219] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:11:06 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653219/ [5653221] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/small-objects} time: 00:22:00 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653221/ [5653222] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} time: 00:12:14 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653222/ [5653223] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:19 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653223/ [5653224] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8} tasks/failover} time: 00:11:13 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653224/ [5653225] rados/standalone/{mon_election/classic supported-random-distro$/{rhel_8} workloads/scrub} time: 01:41:58 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653225/ [5653226] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:06:23 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653226/ [5653227] rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/root mon_election/classic msgr/async start tasks/rados_python} time: 00:23:52 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653227/ [5653228] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:27:11 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653228/ [5653229] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/mapgap thrashosds-health workloads/test_rbd_api} time: 00:20:05 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653229/ [5653230] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/filestore-xfs supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:10:58 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653230/ [5653231] rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{centos_8}} time: 02:37:13 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653231/ [5653232] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/snaps-few-objects-balanced} time: 00:20:03 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653232/ [5653233] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/classic start} time: 00:14:11 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653233/ [5653234] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:11:21 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653234/ [5653235] rados/singleton/{all/ec-lost-unfound mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:29:27 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653235/ [5653236] rados/cephadm/smoke-roleless/{distro/ubuntu_latest start} time: 00:10:47 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653236/ [5653237] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:18:01 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653237/ [5653238] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-balanced} time: 00:25:13 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653238/ [5653239] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:10:03 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653239/ [5653240] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/classic task/test_cephadm_repos} time: 00:07:45 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653240/ [5653241] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_mostlyread} time: 00:32:47 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653241/ [5653242] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:23:33 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653242/ [5653243] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/snaps-few-objects-localized} time: 00:31:50 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653243/ [5653244] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/connectivity start} time: 00:13:35 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653244/ [5653245] rados/singleton/{all/erasure-code-nonregression mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:09:30 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653245/ [5653246] rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} time: 00:39:21 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653246/ [5653247] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/many workloads/rados_5925} time: 00:09:12 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653247/ [5653248] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:36:11 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653248/ [5653249] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:09:32 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653249/ [5653250] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/lockdep} time: 00:20:14 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653250/ [5653251] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:08:31 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653251/ [5653252] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:15:04 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653252/ [5653253] rados/standalone/{mon_election/connectivity supported-random-distro$/{rhel_8} workloads/crush} time: 00:15:20 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653253/ [5653254] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-octopus 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/connectivity thrashosds-health ubuntu_latest} time: 04:58:08 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653254/ [5653255] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:29:59 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653255/ [5653256] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:12 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653256/ [5653257] rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:43:58 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653257/ [5653258] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/snaps-few-objects} time: 00:29:11 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653258/ [5653260] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:11:29 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653260/ [5653261] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:05 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653261/ [5653262] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zstd supported-random-distro$/{rhel_8} tasks/insights} time: 00:13:58 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653262/ [5653263] rados/cephadm/orchestrator_cli/{2-node-mgr orchestrator_cli supported-random-distro$/{rhel_8}} time: 00:13:00 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653263/ [5653264] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:12:50 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653264/ [5653265] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/classic start} time: 00:13:55 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653265/ [5653266] rados/singleton/{all/lost-unfound mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:49:41 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653266/ [5653267] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/write_fadvise_dontneed} time: 00:12:12 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653267/ [5653268] rados/cephadm/smoke-roleless/{distro/centos_8.0 start} time: 00:12:40 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653268/ [5653269] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8} tasks/dashboard} time: 01:11:26 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653269/ [5653270] rados/objectstore/{backends/fusestore supported-random-distro$/{centos_8}} time: 00:09:50 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653270/ [5653271] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{centos_8.0} fixed-2 mon_election/classic} time: 00:17:19 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653271/ [5653272] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:09:17 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653272/ [5653273] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/readwrite} time: 00:10:59 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653273/ [5653274] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:07:49 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653274/ [5653275] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-small-objects-fast-read} time: 00:18:26 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653275/ [5653276] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/none thrashosds-health workloads/radosbench} time: 01:12:16 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653276/ [5653277] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:05:06 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653277/ [5653278] rados/standalone/{mon_election/classic supported-random-distro$/{ubuntu_latest} workloads/erasure-code} time: 00:39:18 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653278/ [5653279] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/admin_socket_objecter_requests} time: 00:36:10 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653279/ [5653280] rados/cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_orch_cli} time: 00:17:04 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653280/ [5653281] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:08:04 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653281/ [5653282] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/connectivity start} time: 00:11:44 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653282/ [5653283] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:10:38 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653283/ [5653284] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:11:43 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653284/ [5653285] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/one workloads/rados_api_tests} time: 00:35:26 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653285/ [5653286] rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_python} time: 00:25:25 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653286/ [5653287] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:26:46 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653287/ [5653288] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/valgrind} time: 00:25:29 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653288/ [5653289] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:23:30 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653289/ [5653290] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/classic start} time: 00:15:33 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653290/ [5653291] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-agent-big} time: 00:25:42 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653291/ [5653292] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:11:06 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653292/ [5653293] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:10:21 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653293/ [5653294] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-hybrid supported-random-distro$/{centos_8} tasks/module_selftest} time: 00:28:38 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653294/ [5653295] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:09:45 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653295/ [5653296] rados/cephadm/smoke-roleless/{distro/centos_latest start} time: 00:10:13 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653296/ [5653297] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/few rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:22:12 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653297/ [5653298] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:10:03 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653298/ [5653299] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/luminous-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/rbd_cls} time: 00:19:52 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653299/ [5653300] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/connectivity start} time: 00:11:33 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653300/ [5653301] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache-agent-small} time: 00:10:20 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653301/ [5653302] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:08:30 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653302/ [5653303] rados/standalone/{mon_election/connectivity supported-random-distro$/{ubuntu_latest} workloads/mgr} time: 00:12:09 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653303/ [5653304] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/repair_test} time: 00:15:41 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653304/ [5653305] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:05:02 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653305/ [5653306] rados/cephadm/with-work/{distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:35:06 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653306/ [5653307] rados/singleton/{all/mon-auth-caps mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:10:08 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653307/ [5653309] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:06:01 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653309/ [5653310] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:25:19 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653310/ [5653311] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-zlib supported-random-distro$/{rhel_8} tasks/dashboard} time: 01:10:00 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653311/ [5653312] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{centos_8}} time: 00:11:15 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653312/ [5653313] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:17:30 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653313/ [5653315] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:20:35 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653315/ [5653316] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:08:02 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653316/ [5653317] rados/singleton/{all/mon-config-key-caps mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:14:15 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653317/ [5653318] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/luminous backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/snaps-few-objects} time: 00:35:31 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653318/ [5653319] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/classic start} time: 00:11:55 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653319/ [5653320] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/rados_mon_osdmap_prune} time: 00:41:34 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653320/ [5653321] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:09:39 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653321/ [5653322] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/lockdep} time: 00:27:06 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653322/ [5653323] rados/cephadm/smoke-roleless/{distro/rhel_8.0 start} time: 00:14:09 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653323/ [5653324] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/cache-pool-snaps} time: 00:24:26 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653324/ [5653325] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:08:44 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653325/ [5653327] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:04 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653327/ [5653328] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:13:25 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653328/ [5653329] rados/singleton/{all/mon-config-keys mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:15:01 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653329/ [5653330] rados/cephadm/smoke/{distro/ubuntu_18.04_podman fixed-2 mon_election/connectivity start} time: 00:13:22 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653330/ [5653331] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8} tasks/progress} time: 00:16:14 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653331/ [5653332] rados/standalone/{mon_election/classic supported-random-distro$/{centos_8} workloads/misc} time: 00:29:59 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653332/ [5653333] rados/cephadm/with-work/{distro/ubuntu_18.04_podman fixed-2 mode/root mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:26:25 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653333/ [5653334] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8} tasks/dashboard} time: 01:08:44 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653334/ [5653335] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} tasks/scrub_test} time: 00:10:32 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653335/ [5653336] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/mimic-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/test_rbd_api} time: 00:18:10 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653336/ [5653337] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-snaps-balanced} time: 00:21:09 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653337/ [5653338] rados/singleton/{all/mon-config mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:10:27 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653338/ [5653339] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:10:49 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653339/ [5653340] rados/cephadm/smoke/{distro/ubuntu_latest fixed-2 mon_election/classic start} time: 00:12:08 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653340/ [5653341] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:37:55 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653341/ [5653342] rados/cephadm/smoke-roleless/{distro/rhel_latest start} time: 00:10:52 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653342/ [5653344] rados/singleton/{all/osd-backfill mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:18:16 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653344/ [5653345] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache-snaps} time: 00:22:30 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653345/ [5653346] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:10:32 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653346/ [5653347] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects} time: 00:18:30 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653347/ [5653348] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:07:31 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653348/ [5653349] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:22:56 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653349/ [5653350] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:10:23 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653350/ [5653351] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/mimic backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:30:09 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653351/ [5653352] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-hybrid supported-random-distro$/{centos_8} tasks/dashboard} time: 01:08:30 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653352/ [5653353] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{ubuntu_latest}} time: 00:30:02 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653353/ [5653354] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/connectivity start} time: 00:13:54 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653354/ [5653355] rados/standalone/{mon_election/connectivity supported-random-distro$/{ubuntu_latest} workloads/mon} time: 00:51:16 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653355/ [5653356] rados/singleton/{all/osd-recovery-incomplete mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:18:25 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653356/ [5653358] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:13:06 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653358/ [5653359] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache} time: 00:12:45 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653359/ [5653360] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:29:58 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653360/ [5653361] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/rados_cls_all validater/valgrind} time: 02:19:20 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653361/ [5653363] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:28 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653363/ [5653364] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:39 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653364/ [5653365] rados/cephadm/with-work/{distro/centos_8.0 fixed-2 mode/packaged mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:26:00 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653365/ [5653366] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} tasks/prometheus} time: 00:09:06 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653366/ [5653368] rados/cephadm/smoke/{distro/centos_latest fixed-2 mon_election/classic start} time: 00:11:31 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653368/ [5653369] rados/singleton/{all/osd-recovery mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:12:11 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653369/ [5653370] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04 start} time: 00:10:24 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653370/ [5653371] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:14:10 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653371/ [5653372] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/pool-snaps-few-objects} time: 00:23:01 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653372/ [5653373] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:08:28 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653373/ [5653374] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} tasks/dashboard} time: 01:08:17 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653374/ [5653375] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat distro$/{ubuntu_18.04} mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/radosbench} time: 01:04:15 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653375/ [5653376] rados/cephadm/workunits/{distro/ubuntu_18.04_podman mon_election/connectivity task/test_cephadm_repos} time: 00:07:32 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653376/ [5653377] rados/singleton/{all/peer mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:06:44 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653377/ [5653378] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:18:31 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653378/ [5653379] rados/cephadm/with-work/{distro/centos_latest fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:33:31 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653379/ [5653380] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/rados_api_tests} time: 00:33:58 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653380/ [5653381] rados/standalone/{mon_election/classic supported-random-distro$/{centos_8} workloads/osd} time: 03:08:29 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653381/ [5653382] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:06:31 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653382/ [5653383] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:20:54 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653383/ [5653384] rados/cephadm/smoke/{distro/rhel_8.0 fixed-2 mon_election/connectivity start} time: 00:15:24 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653384/ [5653385] rados/singleton/{all/pg-autoscaler-progress-off mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:15:02 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653385/ [5653386] rados/cephadm/smoke/{distro/rhel_latest fixed-2 mon_election/classic start} time: 00:11:46 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653386/ [5653387] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:15:16 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653387/ [5653388] rados/cephadm/smoke-roleless/{distro/ubuntu_18.04_podman start} time: 00:11:32 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653388/ [5653389] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/radosbench-high-concurrency} time: 00:51:10 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653389/ [5653390] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-stupid supported-random-distro$/{centos_8} tasks/dashboard} time: 01:09:54 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653390/ [5653391] rados/objectstore/{backends/objectstore supported-random-distro$/{ubuntu_latest}} time: 05:10:00 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653391/ [5653392] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:08 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653392/ [5653393] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:09:25 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653393/ [5653394] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:07:21 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653394/ [5653395] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados tasks/mon_recovery validater/lockdep} time: 00:09:41 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653395/ [5653396] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/snaps-few-objects} time: 00:25:48 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653396/ [5653397] rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on distro$/{ubuntu_18.04} mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/rbd_cls} time: 00:19:24 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653397/ [5653398] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} tasks/rados_cls_all} time: 00:17:32 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653398/ [5653399] rados/singleton/{all/pg-autoscaler mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:15:41 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653399/ [5653400] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:22:18 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653400/ [5653401] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/filestore-xfs supported-random-distro$/{centos_8} tasks/workunits} time: 00:10:51 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653401/ [5653403] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:28:36 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653403/ [5653404] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:10:33 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653404/ [5653405] rados/cephadm/with-work/{distro/rhel_8.0 fixed-2 mode/packaged mon_election/connectivity msgr/async start tasks/rados_python} time: 00:29:07 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653405/ [5653406] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/radosbench} time: 02:23:04 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653406/ [5653407] rados/cephadm/smoke/{distro/ubuntu_18.04 fixed-2 mon_election/connectivity start} time: 00:11:56 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653407/ [5653408] rados/standalone/{mon_election/connectivity supported-random-distro$/{rhel_8} workloads/scrub} time: 01:41:34 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653408/ [5653409] rados/singleton/{all/pg-removal-interruption mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:06:39 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653409/ [5653410] rados/cephadm/workunits/{distro/centos_latest mon_election/classic task/test_orch_cli} time: 00:16:44 info: http://pulpito.ceph.com/teuthology-2020-11-24_07:01:02-rados-master-distro-basic-smithi/5653410/