2020-10-09T14:57:55.482 INFO:root:teuthology version: 1.0.0-2cf4b81 2020-10-09T14:57:55.484 INFO:teuthology.results:subset: '274/9999' 2020-10-09T14:57:55.484 INFO:teuthology.results:seed: '9139' 2020-10-09T15:22:02.723 INFO:root:teuthology version: 1.0.0-2cf4b81 2020-10-09T15:22:02.760 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-10-09T21:37:09.499 INFO:teuthology.results:Tests finished! gathering results... 2020-10-09T21:37:12.351 INFO:scrape:Found 25 jobs 2020-10-09T21:37:13.518 INFO:scrape:Found 7 distinct failure reasons 2020-10-09T21:37:13.519 INFO:scrape:Dead: b'2020-10-09T21:34:53.308 DEBUG:teuthology.exit:Finished running handlers' 2020-10-09T21:37:13.519 INFO:scrape:6 jobs: ['5510471', '5510464', '5510478', '5510468', '5510486', '5510482'] 2020-10-09T21:37:13.520 INFO:scrape:suites intersection: ['ceph', 'clusters/{fixed-2', 'openstack}', 'rados', 'thrashosds-health'] 2020-10-09T21:37:13.520 INFO:scrape:suites union: ['1-pg-log-overrides/normal_pg_log', '1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{default}', '2-recovery-overrides/{more-active-recovery}', '2-recovery-overrides/{more-async-recovery}', 'backoff/normal', 'backoff/peering', 'backoff/peering_and_degraded', 'ceph', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'crc-failures/default', 'd-balancer/crush-compat', 'd-balancer/on', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/osd-delay', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-low-osd-mem-target', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{rhel_8}', 'thrashers/careful', 'thrashers/default', 'thrashers/mapgap', 'thrashers/morepggrow', 'thrashers/pggrow', 'thrashosds-health', 'workloads/cache-snaps}', 'workloads/cache}', 'workloads/pool-snaps-few-objects}', 'workloads/radosbench-high-concurrency}', 'workloads/radosbench}', 'workloads/redirect}'] 2020-10-09T21:37:13.520 INFO:scrape: 2020-10-09T21:37:13.521 INFO:scrape:Failure: "2020-10-09T16:57:51.760959+0000 mon.a (mon.0) 494 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log 2020-10-09T21:37:13.521 INFO:scrape:1 jobs: ['5510467'] 2020-10-09T21:37:13.521 INFO:scrape:suites: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async-v2only', 'objectstore/bluestore-comp-snappy', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_cls_all', 'validater/valgrind}'] 2020-10-09T21:37:13.522 INFO:scrape: 2020-10-09T21:37:13.522 INFO:scrape:Crash: Command failed on smithi162 with status 1: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'truncate --size 0 $TESTDIR/mnt.0/foo'" 2020-10-09T21:37:13.522 INFO:scrape:ceph version 16.0.0-5896-gff268523 (ff268523c2dccc40dc5cd034bded4b745bdb6816) pacific (dev) 1: () 2: (jaeger_tracing::child_span(char const*, std::unique_ptr > const&)+0x36) [0x56380f1a4cd6] 3: (PrimaryLogPG::finish_ctx(PrimaryLogPG::OpContext*, int, int)+0x1fd) [0x56380f30aabd] 4: (PrimaryLogPG::finish_promote(int, PrimaryLogPG::CopyResults*, std::shared_ptr)+0xf48) [0x56380f32f288] 5: (PromoteCallback::finish(boost::tuples::tuple)+0x57) [0x56380f3bdb07] 6: (PrimaryLogPG::process_copy_chunk(hobject_t, unsigned long, int)+0xbac) [0x56380f3421cc] 7: (C_Copyfrom::finish(int)+0xa6) [0x56380f3c0376] 8: (Context::complete(int)+0xd) [0x56380f22611d] 9: (Finisher::finisher_thread_entry()+0x1a5) [0x56380f816305] 10: () 11: clone() 2020-10-09T21:37:13.523 INFO:scrape:1 jobs: ['5510479'] 2020-10-09T21:37:13.523 INFO:scrape:suites: ['mon_election/connectivity', 'rados', 'rados/singleton-nomsgr/{all/cache-fs-trunc', 'supported-random-distro$/{centos_8}}'] 2020-10-09T21:37:13.523 INFO:scrape: 2020-10-09T21:37:13.524 INFO:scrape:Failure: Test failure: test_pool_configuration (tasks.mgr.dashboard.test_pool.PoolTest) 2020-10-09T21:37:13.524 INFO:scrape:1 jobs: ['5510476'] 2020-10-09T21:37:13.524 INFO:scrape:suites: ['debug/mgr', 'mon_election/classic', 'objectstore/bluestore-comp-lz4', 'rados/dashboard/{clusters/{2-node-mgr}', 'supported-random-distro$/{rhel_8}', 'tasks/dashboard}'] 2020-10-09T21:37:13.524 INFO:scrape: 2020-10-09T21:37:13.525 INFO:scrape:Failure: psutil.NoSuchProcess process no longer exists (pid=14787) 2020-10-09T21:37:13.525 INFO:scrape:1 jobs: ['5510466'] 2020-10-09T21:37:13.525 INFO:scrape:suites: ['cluster/{12-osds', 'mon_election/connectivity', 'msgr-failures/osd-delay', 'objectstore/bluestore-comp-snappy', 'openstack}', 'rados', 'rados/thrash-erasure-code-big/{ceph', 'recovery-overrides/{more-partial-recovery}', 'supported-random-distro$/{rhel_8}', 'thrashers/morepggrow', 'thrashosds-health', 'workloads/ec-rados-plugin=jerasure-k=4-m=2}'] 2020-10-09T21:37:13.526 INFO:scrape: 2020-10-09T21:37:13.526 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rados/test.sh 2020-10-09T21:37:13.526 INFO:scrape:1 jobs: ['5510474'] 2020-10-09T21:37:13.527 INFO:scrape:suites: ['1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{more-active-recovery}', 'backoff/normal', 'ceph', 'clusters/{fixed-2', 'crc-failures/default', 'd-balancer/on', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr/async-v2only', 'objectstore/filestore-xfs', 'openstack}', 'rados', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{centos_8}', 'thrashers/none', 'thrashosds-health', 'workloads/rados_api_tests}'] 2020-10-09T21:37:13.527 INFO:scrape: 2020-10-09T21:37:13.527 INFO:scrape:Failure: Command failed (workunit test erasure-code/test-erasure-code.sh) on smithi105 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ff268523c2dccc40dc5cd034bded4b745bdb6816 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh' 2020-10-09T21:37:13.528 INFO:scrape:1 jobs: ['5510484'] 2020-10-09T21:37:13.528 INFO:scrape:suites: ['rados/standalone/{mon_election/classic', 'supported-random-distro$/{centos_8}', 'workloads/erasure-code}'] 2020-10-09T21:37:13.528 INFO:scrape: 2020-10-09T21:37:13.529 INFO:teuthology.results:Sending results to dupadhya@redhat.com: Test Run: ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/ failed: 5 dead: 7 running: 0 waiting: 0 queued: 0 passed: 13 Fail ================================================================= [5510467] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados tasks/rados_cls_all validater/valgrind} ----------------------------------------------------------------- time: 02:47:59 info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510467/ log: http://qa-proxy.ceph.com/teuthology/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510467/ "2020-10-09T16:57:51.760959+0000 mon.a (mon.0) 494 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [5510474] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/rados_api_tests} ----------------------------------------------------------------- time: 03:28:11 info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510474/ log: http://qa-proxy.ceph.com/teuthology/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510474/ sentry: https://sentry.ceph.com/sentry/sepia/?query=777959ccb9b841e19919dee58344df39 Command failed (workunit test rados/test.sh) on smithi094 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ff268523c2dccc40dc5cd034bded4b745bdb6816 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [5510476] rados/dashboard/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} tasks/dashboard} ----------------------------------------------------------------- time: 00:37:10 info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510476/ log: http://qa-proxy.ceph.com/teuthology/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510476/ sentry: https://sentry.ceph.com/sentry/sepia/?query=7ec872ee8d93489f8a86ddc81ef30cfb Test failure: test_pool_configuration (tasks.mgr.dashboard.test_pool.PoolTest) [5510479] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:13:46 info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510479/ log: http://qa-proxy.ceph.com/teuthology/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510479/ sentry: https://sentry.ceph.com/sentry/sepia/?query=4794d08170134d738b16fdbc0cff8ccc Command failed on smithi162 with status 1: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'truncate --size 0 $TESTDIR/mnt.0/foo'" [5510484] rados/standalone/{mon_election/classic supported-random-distro$/{centos_8} workloads/erasure-code} ----------------------------------------------------------------- time: 00:10:20 info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510484/ log: http://qa-proxy.ceph.com/teuthology/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510484/ sentry: https://sentry.ceph.com/sentry/sepia/?query=c05004fd0cd54401ab8f2a733f4f758b Command failed (workunit test erasure-code/test-erasure-code.sh) on smithi105 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ff268523c2dccc40dc5cd034bded4b745bdb6816 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test- erasure-code.sh' Dead ================================================================= [5510464] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-snaps} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510464/ log: http://qa-proxy.ceph.com/teuthology/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510464/ [5510466] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} ----------------------------------------------------------------- time: 06:08:30 info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510466/ log: http://qa-proxy.ceph.com/teuthology/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510466/ psutil.NoSuchProcess process no longer exists (pid=14787) [5510468] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510468/ log: http://qa-proxy.ceph.com/teuthology/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510468/ [5510471] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/pool-snaps-few-objects} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510471/ log: http://qa-proxy.ceph.com/teuthology/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510471/ [5510478] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/radosbench-high-concurrency} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510478/ log: http://qa-proxy.ceph.com/teuthology/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510478/ [5510482] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/radosbench} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510482/ log: http://qa-proxy.ceph.com/teuthology/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510482/ [5510486] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/redirect} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510486/ log: http://qa-proxy.ceph.com/teuthology/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510486/ Pass ================================================================= [5510462] rados/cephadm/dashboard/{distro/centos_latest task/test_e2e} time: 00:15:49 info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510462/ [5510463] rados/singleton/{all/peer mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:05:54 info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510463/ [5510465] rados/cephadm/orchestrator_cli/{2-node-mgr orchestrator_cli supported-random-distro$/{ubuntu_latest}} time: 00:09:13 info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510465/ [5510469] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:11:26 info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510469/ [5510470] rados/cephadm/smoke/{distro/centos_8.0 fixed-2 mon_election/classic start} time: 00:12:39 info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510470/ [5510472] rados/singleton/{all/pg-autoscaler mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:08:09 info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510472/ [5510473] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:08:49 info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510473/ [5510475] rados/cephadm/smoke-roleless/{distro/centos_8.0 start} time: 00:11:17 info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510475/ [5510477] rados/objectstore/{backends/fusestore supported-random-distro$/{rhel_8}} time: 00:09:03 info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510477/ [5510480] rados/cephadm/upgrade/{1-start 2-repo_digest/defaut 3-start-upgrade 4-wait distro$/{centos_latest} fixed-2 mon_election/classic} time: 00:15:14 info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510480/ [5510481] rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/connectivity objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} tasks/prometheus} time: 00:08:28 info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510481/ [5510483] rados/singleton/{all/pg-removal-interruption mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:10:09 info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510483/ [5510485] rados/cephadm/with-work/{distro/rhel_latest fixed-2 mode/root mon_election/classic msgr/async-v2only start tasks/rados_python} time: 00:23:23 info: http://pulpito.ceph.com/ideepika-2020-10-09_14:55:04-rados-wip-jaegertracing-2nd-commit-distro-basic-smithi/5510485/