2023-04-01T00:29:22.008 INFO:root:teuthology version: 0.0.1.dev91+g5338cb9
2023-04-01T00:29:22.009 INFO:teuthology.results:subset: '111/12000'
2023-04-01T00:29:22.009 INFO:teuthology.results:seed: '1840'
2023-04-01T06:55:03.926 INFO:root:teuthology version: 0.0.1.dev91+g5338cb9
2023-04-01T06:55:03.953 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-04-01T09:38:21.287 INFO:teuthology.results:Tests finished! gathering results...
2023-04-01T09:38:50.609 INFO:scrape:Found 387 jobs
2023-04-01T09:38:58.346 INFO:scrape:Found 7 distinct failure reasons
2023-04-01T09:38:58.347 INFO:scrape:Failure: Command failed on smithi072 with status 1: 'sudo yum install -y kernel'
2023-04-01T09:38:58.348 INFO:scrape:6 jobs: ['7228810', '7228864', '7229210', '7229307', '7229226', '7229043']
2023-04-01T09:38:58.348 INFO:scrape:suites intersection: []
2023-04-01T09:38:58.348 INFO:scrape:suites union: ['0-kubeadm', '0-nvme-loop', '1-rook', '2-workload/none', '2-workload/radosbench', 'agent/on', 'ceph', 'cluster/1-node', 'cluster/3-node', 'clusters/{fixed-2', 'd-thrash/default/{default', 'k8s/1.21', 'mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v1only', 'net/calico', 'net/flannel', 'net/host', 'objectstore/bluestore-comp-lz4', 'openstack}', 'rados', 'rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rados/verify/{centos_latest', 'rook/1.7.2}', 'rook/master}', 'task/test_iscsi_pids_limit/{centos_8.stream_container_tools', 'tasks/rados_cls_all', 'test_iscsi_pids_limit}}', 'thrashosds-health}', 'validater/lockdep}']
2023-04-01T09:38:58.349 INFO:scrape:
2023-04-01T09:38:58.349 INFO:scrape:Failure: HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4d/ubuntu/jammy/flavors/default/repo (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1108)')))
2023-04-01T09:38:58.349 INFO:scrape:85 jobs
2023-04-01T09:38:58.350 INFO:scrape:suites intersection: []
2023-04-01T09:38:58.351 INFO:scrape:suites union: ['1-pg-log-overrides/normal_pg_log', '1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{default}', '2-recovery-overrides/{more-active-recovery}', '2-recovery-overrides/{more-async-partial-recovery}', '2-recovery-overrides/{more-async-recovery}', '2-recovery-overrides/{more-partial-recovery}', '3-scrub-overrides/{default}', '3-scrub-overrides/{max-simultaneous-scrubs-2}', '3-scrub-overrides/{max-simultaneous-scrubs-3}', 'backoff/normal', 'backoff/peering', 'backoff/peering_and_degraded', 'ceph', 'cluster/{12-osds', 'clusters/9-mons', 'clusters/{fixed-2', 'clusters/{fixed-4', 'crc-failures/bad_map_crc_failure', 'crc-failures/default', 'd-balancer/crush-compat', 'd-balancer/on', 'debug/mgr', 'fast/fast', 'fast/normal', 'mgr_ttl_cache/disable', 'mgr_ttl_cache/enable', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/many', 'msgr-failures/none', 'msgr-failures/osd-delay', 'msgr-failures/osd-dispatch-delay', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'no_pools', 'objectstore/bluestore-basic-min-osd-mem-target', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-low-osd-mem-target', 'objectstore/bluestore-stupid', 'openstack', 'openstack}', 'rados', 'rados/basic/{ceph', 'rados/mgr/{clusters/{2-node-mgr}', 'rados/monthrash/{ceph', 'rados/multimon/{clusters/3', 'rados/multimon/{clusters/9', 'rados/objectstore/{backends/fusestore', 'rados/objectstore/{backends/objectcacher-stress', 'rados/objectstore/{backends/objectstore-bluestore-b', 'rados/perf/{ceph', 'rados/rest/{mgr-restful', 'rados/singleton-nomsgr/{all/balancer', 'rados/singleton-nomsgr/{all/ceph-post-file', 'rados/singleton-nomsgr/{all/health-warnings', 'rados/singleton-nomsgr/{all/librados_hello_world', 'rados/singleton-nomsgr/{all/msgr', 'rados/singleton-nomsgr/{all/osd_stale_reads', 'rados/singleton-nomsgr/{all/version-number-sanity', 'rados/singleton/{all/divergent_priors', 'rados/singleton/{all/divergent_priors2', 'rados/singleton/{all/erasure-code-nonregression', 'rados/singleton/{all/rebuild-mondb', 'rados/singleton/{all/resolve_stuck_peering', 'rados/singleton/{all/test-crash', 'rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/centos_latest', 'rados/thrash-erasure-code-big/{ceph', 'rados/thrash-erasure-code-isa/{arch/x86_64', 'rados/thrash-erasure-code-overwrites/{bluestore-bitmap', 'rados/thrash-erasure-code-shec/{ceph', 'rados/thrash-erasure-code/{ceph', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'random-objectstore$/{bluestore-bitmap}', 'random-objectstore$/{bluestore-comp-snappy}', 'random-objectstore$/{bluestore-comp-zstd}', 'recovery-overrides/{default}', 'recovery-overrides/{more-active-recovery}', 'recovery-overrides/{more-async-partial-recovery}', 'recovery-overrides/{more-async-recovery}', 'recovery-overrides/{more-partial-recovery}', 'scheduler/dmclock_1Shard_16Threads', 'scheduler/dmclock_default_shards', 'scheduler/wpq_default_shards', 'settings/optimized', 'supported-random-distro$/{ubuntu_latest}', 'supported-random-distro$/{ubuntu_latest}}', 'tasks/failover}', 'tasks/mon_clock_with_skews}', 'tasks/progress}', 'tasks/rados_striper}', 'tasks/rados_workunit_loadgen_mix}', 'tasks/readwrite}', 'tasks/repair_test}', 'tasks/scrub_test}', 'tasks/workunits}', 'test_envlibrados_for_rocksdb}', 'thrashers/careful', 'thrashers/default', 'thrashers/fastread', 'thrashers/mapgap', 'thrashers/minsize_recovery', 'thrashers/morepggrow', 'thrashers/none', 'thrashers/pggrow', 'thrashers/sync-many', 'thrashosds-health', 'ubuntu_latest', 'workloads/admin_socket_objecter_requests}', 'workloads/cache-agent-small}', 'workloads/cache-pool-snaps-readproxy}', 'workloads/cache-pool-snaps}', 'workloads/cache-snaps}', 'workloads/dedup-io-mixed}', 'workloads/dedup-io-snaps}', 'workloads/ec-pool-snaps-few-objects-overwrites}', 'workloads/ec-rados-plugin=isa-k=2-m=1}', 'workloads/ec-rados-plugin=jerasure-k=3-m=1}', 'workloads/ec-rados-plugin=jerasure-k=4-m=2}', 'workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}', 'workloads/ec-rados-plugin=shec-k=4-m=3-c=2}', 'workloads/ec-small-objects-balanced}', 'workloads/ec-small-objects-fast-read-overwrites}', 'workloads/ec-snaps-few-objects-overwrites}', 'workloads/fio_4K_rand_read}', 'workloads/fio_4K_rand_rw}', 'workloads/fio_4M_rand_read}', 'workloads/fio_4M_rand_rw}', 'workloads/fio_4M_rand_write}', 'workloads/pool-snaps-few-objects}', 'workloads/radosbench-high-concurrency}', 'workloads/radosbench_4K_rand_read}', 'workloads/radosbench_4K_seq_read}', 'workloads/radosbench_4M_rand_read}', 'workloads/radosbench_4M_seq_read}', 'workloads/radosbench_4M_write}', 'workloads/radosbench_omap_write}', 'workloads/radosbench}', 'workloads/redirect_promote_tests}', 'workloads/redirect_set_object}', 'workloads/sample_fio}', 'workloads/sample_radosbench}', 'workloads/set-chunks-read}', 'workloads/small-objects-localized}', 'workloads/snaps-few-objects-balanced}', 'workloads/snaps-few-objects-localized}', 'workloads/snaps-few-objects}']
2023-04-01T09:38:58.352 INFO:scrape:
2023-04-01T09:38:58.352 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi158 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c5a0c16ec27f924324bcd2d9df6a44607c0a3b4d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh'
2023-04-01T09:38:58.353 INFO:scrape:2 jobs: ['7229041', '7229305']
2023-04-01T09:38:58.354 INFO:scrape:suites intersection: ['debug/mgr', 'rados/dashboard/{0-single-container-host', 'random-objectstore$/{bluestore-comp-zstd}', 'tasks/e2e}']
2023-04-01T09:38:58.355 INFO:scrape:suites union: ['debug/mgr', 'mon_election/classic', 'mon_election/connectivity', 'rados/dashboard/{0-single-container-host', 'random-objectstore$/{bluestore-comp-zstd}', 'tasks/e2e}']
2023-04-01T09:38:58.355 INFO:scrape:
2023-04-01T09:38:58.356 INFO:scrape:Failure: Test failure: test_cluster_set_user_config_with_non_existing_clusterid (tasks.cephfs.test_nfs.TestNFS)
2023-04-01T09:38:58.356 INFO:scrape:1 jobs: ['7228987']
2023-04-01T09:38:58.357 INFO:scrape:suites: ['agent/on', 'mon_election/connectivity', 'rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun', 'task/test_nfs}']
2023-04-01T09:38:58.358 INFO:scrape:
2023-04-01T09:38:58.358 INFO:scrape:Failure: Command failed (workunit test cephadm/test_repos.sh) on smithi089 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c5a0c16ec27f924324bcd2d9df6a44607c0a3b4d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_repos.sh'
2023-04-01T09:38:58.359 INFO:scrape:1 jobs: ['7228903']
2023-04-01T09:38:58.360 INFO:scrape:suites: ['agent/on', 'mon_election/connectivity', 'rados/cephadm/workunits/{0-distro/ubuntu_20.04', 'task/test_cephadm_repos}']
2023-04-01T09:38:58.360 INFO:scrape:
2023-04-01T09:38:58.361 INFO:scrape:Failure: Command failed (workunit test rados/test.sh) on smithi101 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c5a0c16ec27f924324bcd2d9df6a44607c0a3b4d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh'
2023-04-01T09:38:58.361 INFO:scrape:1 jobs: ['7229265']
2023-04-01T09:38:58.362 INFO:scrape:suites: ['1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{more-partial-recovery}', '3-scrub-overrides/{default}', 'backoff/normal', 'ceph', 'clusters/{fixed-2', 'crc-failures/default', 'd-balancer/on', 'mon_election/connectivity', 'msgr-failures/osd-dispatch-delay', 'msgr/async-v2only', 'objectstore/bluestore-hybrid', 'openstack}', 'rados', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{rhel_8}', 'thrashers/mapgap', 'thrashosds-health', 'workloads/rados_api_tests}']
2023-04-01T09:38:58.362 INFO:scrape:
2023-04-01T09:38:58.362 INFO:scrape:Failure: Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi114 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c5a0c16ec27f924324bcd2d9df6a44607c0a3b4d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh'
2023-04-01T09:38:58.363 INFO:scrape:1 jobs: ['7229167']
2023-04-01T09:38:58.363 INFO:scrape:suites: ['mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v2only', 'objectstore/bluestore-comp-zstd', 'rados', 'rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/rhel_latest', 'supported-random-distro$/{ubuntu_20.04}}', 'test_envlibrados_for_rocksdb}']
2023-04-01T09:38:58.363 INFO:scrape:
2023-04-01T09:38:58.385 INFO:teuthology.results:Sending results to yweinste@radhat.com: Test Run: yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/
failed:  97
dead:    0
running: 0
waiting: 0
queued:  0
passed:  290


Fail
=================================================================
[7228777]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/snaps-few-objects}
-----------------------------------------------------------------
time:   00:07:31
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228777/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228777/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0e6aa96a1935406fa3dcf40d7941ffb6

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228780]  rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:07:27
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228780/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228780/
sentry: https://sentry.ceph.com/organizations/ceph/?query=910d3443916140359219cc6f14ba7c22

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228783]  rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity random-objectstore$/{bluestore-bitmap} supported-random-distro$/{ubuntu_latest} tasks/workunits}
-----------------------------------------------------------------
time:   00:07:28
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228783/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228783/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5e88783f6a2b4ebda8b412adf3c43dc4

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228794]  rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read}
-----------------------------------------------------------------
time:   00:07:33
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228794/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228794/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5e2255caa26146e79b19441f392efcd5

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228796]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-agent-small}
-----------------------------------------------------------------
time:   00:07:32
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228796/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228796/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a4794e59937643b290232ac9d4727a9d

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228806]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/cache-pool-snaps-readproxy}
-----------------------------------------------------------------
time:   00:07:39
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228806/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228806/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7eb1d49194284770985e670d4b9b77a6

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228810]  rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228810/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228810/
sentry: https://sentry.ceph.com/organizations/ceph/?query=cffba14a7a3a4e2baf5bf30fa73829ac

    Command failed on smithi072 with status 1: 'sudo yum install -y kernel'

[7228814]  rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mix}
-----------------------------------------------------------------
time:   00:07:36
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228814/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228814/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7875eef86031496b99b41e6ce817dc10

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228824]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-snaps}
-----------------------------------------------------------------
time:   00:07:42
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228824/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228824/
sentry: https://sentry.ceph.com/organizations/ceph/?query=265af8ffe14d488eb9b6d3f90014abe0

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228840]  rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read}
-----------------------------------------------------------------
time:   00:07:49
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228840/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228840/
sentry: https://sentry.ceph.com/organizations/ceph/?query=37ad4a9450e54e16be5ee34ccd2cf02a

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228848]  rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites}
-----------------------------------------------------------------
time:   00:07:38
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228848/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228848/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0e6933444639489abcf5813a16c8207c

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228854]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/radosbench-high-concurrency}
-----------------------------------------------------------------
time:   00:07:43
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228854/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228854/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5ecc7801b8b2400d8003108f1e0a907a

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228863]  rados/rest/{mgr-restful supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:07:31
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228863/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228863/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8a19eed2394f472b9463fb7b2fb4e0b5

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228864]  rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/calico rook/1.7.2}
-----------------------------------------------------------------
time:   00:06:18
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228864/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228864/
sentry: https://sentry.ceph.com/organizations/ceph/?query=073d607600f14b89940f59dd20c09243

    Command failed on smithi131 with status 1: 'sudo systemctl enable --now
    kubelet && sudo kubeadm config images pull'

[7228871]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/radosbench}
-----------------------------------------------------------------
time:   00:07:28
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228871/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228871/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4c4981e5ead24d4baa46428450a6b4a9

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228879]  rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
-----------------------------------------------------------------
time:   00:07:48
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228879/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228879/
sentry: https://sentry.ceph.com/organizations/ceph/?query=84bbcd1401414cb79738d966259d0c5e

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228887]  rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2}
-----------------------------------------------------------------
time:   00:07:56
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228887/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228887/
sentry: https://sentry.ceph.com/organizations/ceph/?query=40c9722a18f142489e485c49a1ced140

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228898]  rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read}
-----------------------------------------------------------------
time:   00:07:40
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228898/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228898/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c2b04f48e77b47608f06bf26bcf6f8f7

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228903]  rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_cephadm_repos}
-----------------------------------------------------------------
time:   00:06:49
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228903/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228903/
sentry: https://sentry.ceph.com/organizations/ceph/?query=dc6741c289f34d57a1462f2f748f90d5

    Command failed (workunit test cephadm/test_repos.sh) on smithi089 with
    status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c5a0c16ec27f924324bcd2d9df6a44607c0a3b4d
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_repos.sh'

[7228908]  rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/sync-many workloads/snaps-few-objects}
-----------------------------------------------------------------
time:   00:07:37
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228908/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228908/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9c09959c9d16489c8707903ac46d85a0

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228910]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-balanced}
-----------------------------------------------------------------
time:   00:07:31
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228910/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228910/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a975054011884c238dd172922491a39c

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228917]  rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/readwrite}
-----------------------------------------------------------------
time:   00:07:39
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228917/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228917/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d35c3d42888d44bfa41a923e6064ca6a

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228927]  rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_latest} tasks/failover}
-----------------------------------------------------------------
time:   00:07:32
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228927/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228927/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7552513fad154451915b33da461914fd

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228933]  rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:07:38
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228933/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228933/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7deab4b8724846deb74f013c8164978b

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228943]  rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read}
-----------------------------------------------------------------
time:   00:07:26
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228943/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228943/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9568b9fd67384691a10d2031b14c4b6d

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228951]  rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites}
-----------------------------------------------------------------
time:   00:07:44
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228951/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228951/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4855a6d53008464b88f4f2d632d154f3

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228959]  rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:07:32
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228959/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228959/
sentry: https://sentry.ceph.com/organizations/ceph/?query=10c64fef9f2f4df4a2652295af5efc80

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228965]  rados/objectstore/{backends/fusestore supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:07:31
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228965/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228965/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2023b4f713fc42be938cd5ca127248d7

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228979]  rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/repair_test}
-----------------------------------------------------------------
time:   00:07:36
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228979/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228979/
sentry: https://sentry.ceph.com/organizations/ceph/?query=86ffb256679a441fbbd8ac6a929197a9

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228987]  rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_nfs}
-----------------------------------------------------------------
time:   00:19:10
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228987/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228987/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8ac8bdb260bc4ed79916127b2d6f13ca

    Test failure: test_cluster_set_user_config_with_non_existing_clusterid
    (tasks.cephfs.test_nfs.TestNFS)

[7228991]  rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write}
-----------------------------------------------------------------
time:   00:07:47
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228991/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228991/
sentry: https://sentry.ceph.com/organizations/ceph/?query=eca706790c28413eb7dd5a234743fa50

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228995]  rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:07:40
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228995/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228995/
sentry: https://sentry.ceph.com/organizations/ceph/?query=26d3300bd07247ab9b0517eaf5fe699f

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7228997]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/dedup-io-snaps}
-----------------------------------------------------------------
time:   00:07:38
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228997/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228997/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9c6e5194408f4f4ca1addc4af3be825b

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229003]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/pool-snaps-few-objects}
-----------------------------------------------------------------
time:   00:07:42
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229003/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229003/
sentry: https://sentry.ceph.com/organizations/ceph/?query=34a72a0fa15b4c7ab507d62a49d7ebef

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229007]  rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:07:31
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229007/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229007/
sentry: https://sentry.ceph.com/organizations/ceph/?query=34d8f02b6f4b4cbcb61d2683186815c7

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229021]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/radosbench}
-----------------------------------------------------------------
time:   00:07:31
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229021/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229021/
sentry: https://sentry.ceph.com/organizations/ceph/?query=579034a2377f4fae897b01b6c945ad05

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229025]  rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
-----------------------------------------------------------------
time:   00:07:46
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229025/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229025/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9d3eae0bd40742efa0f11ca65ed45c15

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229033]  rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/scrub_test}
-----------------------------------------------------------------
time:   00:07:41
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229033/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229033/
sentry: https://sentry.ceph.com/organizations/ceph/?query=3cff4d7b6dea4cf6b736a690b56e09ff

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229039]  rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2}
-----------------------------------------------------------------
time:   00:07:45
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229039/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229039/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f982382749a04b2aa1ebffd7ac05b18f

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229041]  rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-comp-zstd} tasks/e2e}
-----------------------------------------------------------------
time:   00:28:27
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229041/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229041/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a1202b5f10054ebbba0d9a2ae02c47b8

    Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi158
    with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd
    -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c5a0c16ec27f924324bcd2d9df6a44607c0a3b4d
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh'

[7229043]  rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/flannel rook/master}
-----------------------------------------------------------------
time:   00:06:33
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229043/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229043/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9e156b8fe8e0407ebb209ecd7ccaf3ab

    Command failed on smithi031 with status 1: 'sudo systemctl enable --now
    kubelet && sudo kubeadm config images pull'

[7229049]  rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write}
-----------------------------------------------------------------
time:   00:07:31
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229049/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229049/
sentry: https://sentry.ceph.com/organizations/ceph/?query=008b7342e5a345feb5ac7773b3fcf96f

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229053]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/set-chunks-read}
-----------------------------------------------------------------
time:   00:07:35
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229053/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229053/
sentry: https://sentry.ceph.com/organizations/ceph/?query=85310b3f03214b79a1400f630f220299

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229059]  rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/centos_latest test_envlibrados_for_rocksdb} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:07:34
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229059/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229059/
sentry: https://sentry.ceph.com/organizations/ceph/?query=994a355b3f704d489b92cc45c3f49dec

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229085]  rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:07:39
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229085/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229085/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b1291912efac44628302d331292a8773

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229097]  rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio}
-----------------------------------------------------------------
time:   00:07:36
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229097/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229097/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a90f45944ae2427cb7aa30fbc1669295

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229103]  rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1}
-----------------------------------------------------------------
time:   00:07:42
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229103/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229103/
sentry: https://sentry.ceph.com/organizations/ceph/?query=21666b2b8d264e2ca4e70888863fc8ff

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229105]  rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:07:34
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229105/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229105/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7c7dfc6bef54469bb5d1fc7fe207e81a

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229115]  rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
-----------------------------------------------------------------
time:   00:07:33
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229115/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229115/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d0701f31377a432bab8b83fb18e807b7

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229151]  rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:07:28
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229151/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229151/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0ff44cc1333445b4876ebeb12a8fa7bc

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229156]  rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench}
-----------------------------------------------------------------
time:   00:07:42
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229156/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229156/
sentry: https://sentry.ceph.com/organizations/ceph/?query=81a02edf058144feaf44e67c572be840

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229158]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/dedup-io-mixed}
-----------------------------------------------------------------
time:   00:07:42
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229158/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229158/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f06111dd27d44d7a9d657d4a21bfb9ee

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229165]  rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites}
-----------------------------------------------------------------
time:   00:07:39
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229165/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229165/
sentry: https://sentry.ceph.com/organizations/ceph/?query=69762409558d445fae52e436bc8f194e

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229167]  rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/rhel_latest test_envlibrados_for_rocksdb} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04}}
-----------------------------------------------------------------
time:   00:09:58
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229167/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229167/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7da68d64918c4c998ffecb91afc1d9ff

    Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on
    smithi114 with status 2: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c5a0c16ec27f924324bcd2d9df6a44607c0a3b4d
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh'

[7229177]  rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{ubuntu_latest} tasks/progress}
-----------------------------------------------------------------
time:   00:07:53
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229177/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229177/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7df14243251e4a4fabbab80d84f0f039

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229179]  rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:07:37
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229179/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229179/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f499982384d74559b44bd70321558139

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229191]  rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
-----------------------------------------------------------------
time:   00:07:42
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229191/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229191/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7828749512e544a38dfe0c82d76ac997

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229192]  rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
-----------------------------------------------------------------
time:   00:07:49
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229192/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229192/
sentry: https://sentry.ceph.com/organizations/ceph/?query=753f8710823e4505af779d4a58085331

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229194]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/redirect_promote_tests}
-----------------------------------------------------------------
time:   00:07:43
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229194/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229194/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1ba7d94935484161a9e28b1121fd725f

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229196]  rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read}
-----------------------------------------------------------------
time:   00:07:35
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229196/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229196/
sentry: https://sentry.ceph.com/organizations/ceph/?query=989e60ddb1f343a5affed7933eec06af

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229199]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/redirect_set_object}
-----------------------------------------------------------------
time:   00:07:40
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229199/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229199/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ad1314a6c71c43e19aefdfe8568a8a1b

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229200]  rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:07:43
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229200/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229200/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5330f189108043eb848fd2ad69eed204

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229203]  rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews}
-----------------------------------------------------------------
time:   00:07:41
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229203/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229203/
sentry: https://sentry.ceph.com/organizations/ceph/?query=120fa6e94b274cdaaad9349ab6b4606b

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229207]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/small-objects-localized}
-----------------------------------------------------------------
time:   00:07:38
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229207/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229207/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b6fb491260a94b1f8725fa2a792b6ead

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229210]  rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/host rook/1.7.2}
-----------------------------------------------------------------
time:   00:06:15
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229210/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229210/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1763ec2ee1df439a9ba6416415efa3cd

    Command failed on smithi008 with status 1: 'sudo systemctl enable --now
    kubelet && sudo kubeadm config images pull'

[7229215]  rados/singleton/{all/divergent_priors mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:07:44
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229215/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229215/
sentry: https://sentry.ceph.com/organizations/ceph/?query=95bf1ba04efd42e8a40bb4462f8af56a

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229219]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/snaps-few-objects-localized}
-----------------------------------------------------------------
time:   00:07:44
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229219/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229219/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5cfad684224c4229b269b1cdb2a1cb7f

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229220]  rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:07:37
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229220/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229220/
sentry: https://sentry.ceph.com/organizations/ceph/?query=3a34529f16724d4ea643201663752b09

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229221]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/snaps-few-objects}
-----------------------------------------------------------------
time:   00:07:44
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229221/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229221/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9a8d5419acf1473299e78b6782a7ebc1

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229222]  rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw}
-----------------------------------------------------------------
time:   00:07:35
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229222/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229222/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1067f2c359b74a4d958550a36fcf0b1d

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229223]  rados/singleton/{all/divergent_priors2 mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:07:37
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229223/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229223/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f04ffa2efcf0465ebd29f42360519756

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229226]  rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_iscsi_pids_limit/{centos_8.stream_container_tools test_iscsi_pids_limit}}
-----------------------------------------------------------------
time:   00:07:27
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229226/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229226/
sentry: https://sentry.ceph.com/organizations/ceph/?query=01cdcdd13ee74a89b5914472ac2127e3

    Command failed on smithi177 with status 1: 'TESTDIR=/home/ubuntu/cephtest
    bash -s'

[7229228]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/admin_socket_objecter_requests}
-----------------------------------------------------------------
time:   00:07:44
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229228/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229228/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0b390798ddf045798b95df65fb40a935

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229229]  rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-small-objects-balanced}
-----------------------------------------------------------------
time:   00:07:39
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229229/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229229/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e1b31af04051458a86cfc790ae9f222e

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229251]  rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read}
-----------------------------------------------------------------
time:   00:07:31
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229251/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229251/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9919a84c33264bb7a602167f64f7c7dd

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229255]  rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:07:34
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229255/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229255/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a61610477473421cbb6226b5bbbda975

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229259]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/dedup-io-snaps}
-----------------------------------------------------------------
time:   00:07:42
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229259/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229259/
sentry: https://sentry.ceph.com/organizations/ceph/?query=65b6fbef1001401ca8126f6fc29ad482

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229262]  rados/singleton/{all/erasure-code-nonregression mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:07:37
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229262/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229262/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f994af04838b48c1845898e88055ac3d

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229265]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/rados_api_tests}
-----------------------------------------------------------------
time:   01:03:08
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229265/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229265/
sentry: https://sentry.ceph.com/organizations/ceph/?query=80013df3b7564523b5b52bc34f1d4de6

    Command failed (workunit test rados/test.sh) on smithi101 with status 1:
    'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c5a0c16ec27f924324bcd2d9df6a44607c0a3b4d
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh'

[7229276]  rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw}
-----------------------------------------------------------------
time:   00:07:31
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229276/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229276/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2d390cc761a84dc58894e2c02193b48f

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229286]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/set-chunks-read}
-----------------------------------------------------------------
time:   00:07:49
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229286/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229286/
sentry: https://sentry.ceph.com/organizations/ceph/?query=99bf03c2404646ecb8ba4caaf0893f85

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229287]  rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/rados_striper}
-----------------------------------------------------------------
time:   00:07:39
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229287/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229287/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d304094c4aff4fa990fd2bd4e9897fda

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229295]  rados/objectstore/{backends/objectstore-bluestore-b supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:07:34
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229295/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229295/
sentry: https://sentry.ceph.com/organizations/ceph/?query=178e295d80984e12b97d6ac245b95ef7

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229297]  rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:07:40
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229297/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229297/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7a9c1f2eb25f46da91a934bd6536307d

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229303]  rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write}
-----------------------------------------------------------------
time:   00:07:31
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229303/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229303/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a9fc81bf7443436cb2f43951aefd19e5

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229305]  rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-zstd} tasks/e2e}
-----------------------------------------------------------------
time:   00:28:24
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229305/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229305/
sentry: https://sentry.ceph.com/organizations/ceph/?query=db30b8d8c5d84e9c923a02197a6589ff

    Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi131
    with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd
    -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c5a0c16ec27f924324bcd2d9df6a44607c0a3b4d
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh'

[7229307]  rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/calico rook/master}
-----------------------------------------------------------------
time:   00:06:39
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229307/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229307/
sentry: https://sentry.ceph.com/organizations/ceph/?query=fedef3f7f9054723b0895de86f9e29f0

    Command failed on smithi050 with status 1: 'sudo systemctl enable --now
    kubelet && sudo kubeadm config images pull'

[7229312]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/admin_socket_objecter_requests}
-----------------------------------------------------------------
time:   00:07:36
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229312/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229312/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f40d2e5e4e344b7a8c93f9bbc1068faa

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229320]  rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
-----------------------------------------------------------------
time:   00:07:44
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229320/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229320/
sentry: https://sentry.ceph.com/organizations/ceph/?query=77dcd17b5b924f5f8380e0b3f8c9e3a5

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229321]  rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
-----------------------------------------------------------------
time:   00:07:54
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229321/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229321/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8ad504ab59064d9a8e354f9c87fd2a91

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229325]  rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}
-----------------------------------------------------------------
time:   00:07:49
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229325/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229325/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e85868ae8d6c4f4e8fb9882fa68af413

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229326]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps}
-----------------------------------------------------------------
time:   00:07:40
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229326/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229326/
sentry: https://sentry.ceph.com/organizations/ceph/?query=922c388df2b74f1fbd4cf24d121c7741

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229330]  rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read}
-----------------------------------------------------------------
time:   00:07:35
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229330/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229330/
sentry: https://sentry.ceph.com/organizations/ceph/?query=45efdcfd357940319338f30c1e258005

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229332]  rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews}
-----------------------------------------------------------------
time:   00:07:50
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229332/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229332/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8c33f3e9b22e4da2a368fdece6f5b24e

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229338]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/dedup-io-mixed}
-----------------------------------------------------------------
time:   00:07:41
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229338/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229338/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b4485a52e5034c0992b00f48bf5898d9

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229341]  rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:07:44
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229341/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229341/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f4970c11c071469b809c5135e4c1897e

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))

[7229342]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/dedup-io-snaps}
-----------------------------------------------------------------
time:   00:07:39
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229342/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229342/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0a248594525d41dd850576ce8759b984

    HTTPSConnectionPool(host='4.chacra.ceph.com', port=443): Max retries
    exceeded with url: /repos/ceph/reef/c5a0c16ec27f924324bcd2d9df6a44607c0a3b4
    d/ubuntu/jammy/flavors/default/repo (Caused by
    SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
    certificate verify failed: certificate has expired (_ssl.c:1108)')))



Pass
=================================================================
[7228778] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/write_fadvise_dontneed}
time:   00:19:49
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228778/

[7228779] rados/singleton/{all/mon-config mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}}
time:   00:16:25
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228779/

[7228781] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/scrub}
time:   02:47:55
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228781/

[7228782] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/classic start}
time:   00:17:53
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228782/

[7228784] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/admin_socket_objecter_requests}
time:   00:26:20
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228784/

[7228786] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-small-objects-many-deletes}
time:   00:24:43
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228786/

[7228788] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-agent-big}
time:   00:34:30
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228788/

[7228790] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:29:52
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228790/

[7228792] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:12:20
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228792/

[7228798] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}}
time:   00:18:57
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228798/

[7228800] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add}
time:   00:18:15
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228800/

[7228802] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}
time:   00:15:53
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228802/

[7228804] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{centos_8}}
time:   00:12:11
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228804/

[7228808] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} tasks/mon_recovery}
time:   00:23:17
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228808/

[7228812] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} thrashers/none thrashosds-health workloads/cache-pool-snaps}
time:   00:19:54
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228812/

[7228816] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_adoption}
time:   00:14:29
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228816/

[7228818] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}}
time:   00:53:06
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228818/

[7228820] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache-snaps-balanced}
time:   00:27:52
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228820/

[7228822] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{centos_8}}
time:   00:31:00
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228822/

[7228826] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/cache}
time:   00:16:11
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228826/

[7228828] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start}
time:   00:24:10
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228828/

[7228830] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}}
time:   00:23:47
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228830/

[7228832] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/dedup-io-mixed}
time:   00:13:22
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228832/

[7228834] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} thrashers/one workloads/rados_mon_workunits}
time:   00:30:53
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228834/

[7228836] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{rhel_8}}
time:   00:15:50
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228836/

[7228838] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/dedup-io-snaps}
time:   00:12:50
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228838/

[7228842] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag}
time:   00:15:35
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228842/

[7228844] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/pacific backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/test_rbd_api}
time:   00:22:39
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228844/

[7228846] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} thrashers/none thrashosds-health workloads/pool-snaps-few-objects}
time:   00:25:45
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228846/

[7228850] rados/singleton/{all/peer mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}}
time:   00:13:08
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228850/

[7228852] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/rados_api_tests}
time:   00:28:23
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228852/

[7228856] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_cephadm}
time:   00:19:12
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228856/

[7228858] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-bitmap} tasks/dashboard}
time:   01:08:24
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228858/

[7228860] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{centos_8} tasks/crash}
time:   00:14:11
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228860/

[7228862] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{ubuntu_20.04}}
time:   00:39:40
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228862/

[7228865] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{centos_8}}
time:   00:21:24
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228865/

[7228866] rados/standalone/{supported-random-distro$/{centos_8} workloads/c2c}
time:   00:17:45
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228866/

[7228867] rados/upgrade/parallel/{0-random-distro$/{rhel_8.6_container_tools_3.0} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}}
time:   01:26:54
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228867/

[7228868] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest}
time:   00:21:21
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228868/

[7228869] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-small-objects}
time:   00:24:36
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228869/

[7228873] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} tasks/rados_workunit_loadgen_mostlyread}
time:   00:30:38
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228873/

[7228875] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}}
time:   00:19:03
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228875/

[7228877] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:28:07
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228877/

[7228881] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/redirect}
time:   00:23:20
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228881/

[7228883] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:16:44
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228883/

[7228885] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/redirect_promote_tests}
time:   00:13:51
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228885/

[7228889] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}}
time:   00:10:17
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228889/

[7228891] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/redirect_set_object}
time:   00:14:52
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228891/

[7228893] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04}}
time:   00:14:34
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228893/

[7228894] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews}
time:   00:11:22
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228894/

[7228895] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind}
time:   00:30:35
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228895/

[7228896] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/set-chunks-read}
time:   00:16:40
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228896/

[7228897] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/basic 3-final}
time:   00:14:02
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228897/

[7228899] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/small-objects-balanced}
time:   00:26:07
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228899/

[7228900] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{ubuntu_20.04}}
time:   00:11:54
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228900/

[7228901] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/small-objects-localized}
time:   00:23:10
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228901/

[7228902] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04}}
time:   00:09:05
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228902/

[7228906] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/small-objects}
time:   00:21:39
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228906/

[7228911] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}}
time:   00:09:35
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228911/

[7228914] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/snaps-few-objects-localized}
time:   00:28:13
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228914/

[7228915] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait}
time:   00:15:02
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228915/

[7228919] rados/singleton/{all/radostool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04}}
time:   00:14:50
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228919/

[7228921] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/snaps-few-objects}
time:   00:30:41
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228921/

[7228923] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/write_fadvise_dontneed}
time:   00:19:43
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228923/

[7228925] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:17:42
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228925/

[7228929] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04}}
time:   00:20:59
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228929/

[7228931] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04}}
time:   00:32:04
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228931/

[7228935] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/crush}
time:   00:20:55
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228935/

[7228937] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/admin_socket_objecter_requests}
time:   00:33:58
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228937/

[7228939] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2}
time:   00:24:07
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228939/

[7228941] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/cache-snaps}
time:   00:36:48
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228941/

[7228945] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-agent-big}
time:   01:18:45
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228945/

[7228947] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:27:24
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228947/

[7228949] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:12:23
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228949/

[7228953] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-agent-small}
time:   00:17:53
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228953/

[7228955] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_iscsi_pids_limit/{centos_8.stream_container_tools test_iscsi_pids_limit}}
time:   00:16:54
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228955/

[7228957] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}
time:   00:14:30
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228957/

[7228961] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-pool-snaps-readproxy}
time:   00:22:45
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228961/

[7228963] rados/singleton-nomsgr/{all/crushdiff mon_election/connectivity rados supported-random-distro$/{centos_8}}
time:   00:12:22
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228963/

[7228967] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps}
time:   00:24:05
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228967/

[7228969] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} tasks/mon_clock_with_skews}
time:   00:08:25
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228969/

[7228971] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep}
time:   00:29:30
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228971/

[7228973] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate}
time:   00:16:49
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228973/

[7228975] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache-snaps-balanced}
time:   00:28:54
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228975/

[7228977] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04}}
time:   00:54:28
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228977/

[7228981] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-snaps}
time:   00:26:09
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228981/

[7228983] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{ubuntu_20.04}}
time:   00:08:55
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228983/

[7228985] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache}
time:   00:14:40
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228985/

[7228989] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/dedup-io-mixed}
time:   00:12:39
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228989/

[7228993] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04} thrashers/sync workloads/pool-create-delete}
time:   00:15:27
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228993/

[7228999] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:17:52
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7228999/

[7229001] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{rhel_8}}
time:   00:13:31
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229001/

[7229005] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/rados_api_tests}
time:   00:30:06
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229005/

[7229009] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{rhel_8} tasks/insights}
time:   00:17:46
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229009/

[7229011] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all}
time:   00:15:41
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229011/

[7229013] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/radosbench-high-concurrency}
time:   00:36:03
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229013/

[7229015] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1}
time:   00:29:59
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229015/

[7229017] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{ubuntu_20.04}}
time:   00:10:20
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229017/

[7229019] rados/standalone/{supported-random-distro$/{centos_8} workloads/erasure-code}
time:   01:28:38
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229019/

[7229023] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:27:08
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229023/

[7229027] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/redirect}
time:   00:15:32
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229027/

[7229029] rados/singleton/{all/test-noautoscale-flag mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}}
time:   00:12:05
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229029/

[7229031] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_orch_cli}
time:   00:18:18
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229031/

[7229035] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/redirect_promote_tests}
time:   00:15:01
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229035/

[7229037] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/quincy backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/radosbench}
time:   01:45:32
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229037/

[7229045] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{centos_8}}
time:   00:12:40
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229045/

[7229047] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/redirect_set_object}
time:   00:17:43
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229047/

[7229051] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:18:22
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229051/

[7229055] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} tasks/mon_recovery}
time:   00:15:37
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229055/

[7229057] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind}
time:   02:04:26
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229057/

[7229061] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites}
time:   00:25:34
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229061/

[7229063] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/small-objects-balanced}
time:   00:26:50
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229063/

[7229065] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/small-objects-localized}
time:   00:21:54
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229065/

[7229067] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{centos_8}}
time:   00:13:24
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229067/

[7229069] rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0} 1-start 2-services/rgw 3-final}
time:   00:16:01
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229069/

[7229071] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/small-objects}
time:   00:21:18
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229071/

[7229073] rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04}}
time:   00:52:42
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229073/

[7229075] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{ubuntu_20.04}}
time:   00:16:45
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229075/

[7229077] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} thrashers/none thrashosds-health workloads/snaps-few-objects-balanced}
time:   00:25:28
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229077/

[7229079] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} thrashers/force-sync-many workloads/rados_5925}
time:   00:11:35
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229079/

[7229081] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_orch_cli_mon}
time:   00:28:20
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229081/

[7229083] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/snaps-few-objects-localized}
time:   00:30:38
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229083/

[7229087] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}}
time:   00:38:03
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229087/

[7229089] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/snaps-few-objects}
time:   00:27:31
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229089/

[7229091] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/libcephsqlite}
time:   00:16:41
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229091/

[7229093] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/write_fadvise_dontneed}
time:   00:17:35
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229093/

[7229095] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{centos_8} tasks/module_selftest}
time:   00:30:02
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229095/

[7229099] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add}
time:   00:15:46
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229099/

[7229101] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/admin_socket_objecter_requests}
time:   00:20:41
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229101/

[7229107] rados/standalone/{supported-random-distro$/{centos_8} workloads/mgr}
time:   00:18:11
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229107/

[7229109] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest}
time:   00:21:52
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229109/

[7229111] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}}
time:   00:23:46
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229111/

[7229113] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/cache-agent-big}
time:   00:19:57
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229113/

[7229117] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:15:15
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229117/

[7229119] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:23:34
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229119/

[7229121] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache-agent-small}
time:   00:15:04
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229121/

[7229123] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04}}
time:   00:32:49
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229123/

[7229125] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps-readproxy}
time:   00:23:56
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229125/

[7229127] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}
time:   00:14:30
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229127/

[7229129] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}}
time:   00:24:11
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229129/

[7229131] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}}
time:   00:21:05
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229131/

[7229133] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/cache-pool-snaps}
time:   00:29:54
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229133/

[7229135] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_adoption}
time:   00:10:06
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229135/

[7229137] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} tasks/mon_clock_no_skews}
time:   00:09:10
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229137/

[7229139] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep}
time:   00:16:05
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229139/

[7229141] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/cache-snaps-balanced}
time:   00:22:32
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229141/

[7229143] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/rbd_cls}
time:   00:22:51
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229143/

[7229145] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-snaps}
time:   00:26:21
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229145/

[7229147] rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}}
time:   00:12:55
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229147/

[7229149] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag}
time:   00:18:12
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229149/

[7229153] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/cache}
time:   00:13:29
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229153/

[7229155] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04} tasks/rados_api_tests}
time:   00:27:56
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229155/

[7229160] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/many workloads/rados_api_tests}
time:   00:36:34
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229160/

[7229163] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/dedup-io-snaps}
time:   00:16:44
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229163/

[7229169] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:17:21
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229169/

[7229171] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/pool-snaps-few-objects}
time:   00:28:14
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229171/

[7229173] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}}
time:   00:09:48
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229173/

[7229175] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/rados_api_tests}
time:   00:32:40
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229175/

[7229181] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_cephadm}
time:   00:18:40
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229181/

[7229183] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/radosbench-high-concurrency}
time:   00:28:07
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229183/

[7229185] rados/singleton/{all/admin-socket mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}}
time:   00:13:03
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229185/

[7229186] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-radosbench}
time:   01:20:45
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229186/

[7229187] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/radosbench}
time:   00:55:33
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229187/

[7229188] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{centos_8}}
time:   00:31:01
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229188/

[7229189] rados/standalone/{supported-random-distro$/{rhel_8} workloads/misc}
time:   01:10:22
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229189/

[7229190] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} thrashers/none thrashosds-health workloads/redirect}
time:   00:12:08
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229190/

[7229193] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait}
time:   00:15:24
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229193/

[7229195] rados/singleton/{all/backfill-toofull mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}}
time:   00:24:18
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229195/

[7229197] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2}
time:   00:28:21
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229197/

[7229198] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} tasks/rados_cls_all}
time:   00:19:40
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229198/

[7229201] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:16:52
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229201/

[7229202] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/set-chunks-read}
time:   00:18:54
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229202/

[7229204] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind}
time:   00:48:20
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229204/

[7229205] rados/singleton/{all/deduptool mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04}}
time:   00:10:30
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229205/

[7229206] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/small-objects-balanced}
time:   00:24:52
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229206/

[7229208] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_cephadm_repos}
time:   00:10:06
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229208/

[7229209] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-comp-zlib} tasks/dashboard}
time:   01:07:30
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229209/

[7229211] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{centos_8}}
time:   00:21:26
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229211/

[7229212] rados/upgrade/parallel/{0-random-distro$/{rhel_8.6_container_tools_rhel8} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}}
time:   01:26:28
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229212/

[7229213] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/small-objects}
time:   00:22:57
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229213/

[7229214] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/snaps-few-objects}
time:   00:39:40
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229214/

[7229216] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/snaps-few-objects-balanced}
time:   00:29:16
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229216/

[7229217] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} thrashers/one workloads/rados_mon_osdmap_prune}
time:   00:58:25
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229217/

[7229218] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools} 1-start 2-services/basic 3-final}
time:   00:14:32
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229218/

[7229224] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{centos_8} tasks/prometheus}
time:   00:15:12
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229224/

[7229225] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/write_fadvise_dontneed}
time:   00:15:17
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229225/

[7229227] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/rados_python}
time:   00:17:50
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229227/

[7229230] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites}
time:   00:40:39
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229230/

[7229231] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} thrashers/none thrashosds-health workloads/cache-agent-big}
time:   00:16:45
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229231/

[7229232] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}}
time:   00:13:22
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229232/

[7229233] rados/standalone/{supported-random-distro$/{rhel_8} workloads/mon}
time:   01:21:38
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229233/

[7229234] rados/singleton/{all/dump-stuck mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}}
time:   00:13:25
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229234/

[7229235] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:28:44
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229235/

[7229236] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:12:11
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229236/

[7229237] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rmdir-reactivate}
time:   00:15:50
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229237/

[7229238] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache-agent-small}
time:   00:18:14
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229238/

[7229239] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/cache-pool-snaps-readproxy}
time:   00:30:31
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229239/

[7229240] rados/objectstore/{backends/objectstore-bluestore-a supported-random-distro$/{ubuntu_20.04}}
time:   02:16:30
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229240/

[7229241] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}
time:   00:15:57
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229241/

[7229242] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/cache-pool-snaps}
time:   00:26:40
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229242/

[7229243] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:18:11
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229243/

[7229244] rados/singleton/{all/ec-inconsistent-hinfo mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}}
time:   00:52:32
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229244/

[7229245] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{rhel_8}}
time:   00:14:04
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229245/

[7229246] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-snaps-balanced}
time:   00:26:18
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229246/

[7229247] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/mon_recovery}
time:   00:15:57
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229247/

[7229248] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados tasks/rados_cls_all validater/lockdep}
time:   00:21:24
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229248/

[7229249] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}}
time:   00:36:27
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229249/

[7229250] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-snaps}
time:   00:24:19
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229250/

[7229252] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_nfs}
time:   00:31:52
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229252/

[7229253] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} thrashers/none thrashosds-health workloads/cache}
time:   00:14:30
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229253/

[7229254] rados/singleton/{all/ec-lost-unfound mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04}}
time:   00:48:24
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229254/

[7229256] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/dedup-io-mixed}
time:   00:15:00
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229256/

[7229257] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} tasks/rados_stress_watch}
time:   00:18:43
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229257/

[7229258] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/sync-many workloads/rados_mon_workunits}
time:   00:33:06
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229258/

[7229260] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/repave-all}
time:   00:14:59
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229260/

[7229261] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/pool-snaps-few-objects}
time:   00:34:39
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229261/

[7229263] rados/singleton-nomsgr/{all/crushdiff mon_election/classic rados supported-random-distro$/{rhel_8}}
time:   00:13:30
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229263/

[7229264] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/test_rbd_api}
time:   00:20:40
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229264/

[7229266] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{rhel_8} tasks/workunits}
time:   00:13:54
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229266/

[7229267] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:17:12
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229267/

[7229268] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/radosbench-high-concurrency}
time:   00:38:04
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229268/

[7229269] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-small-objects-fast-read}
time:   00:23:37
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229269/

[7229270] rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}}
time:   00:53:16
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229270/

[7229271] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/radosbench}
time:   00:49:20
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229271/

[7229272] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{centos_8}}
time:   00:11:50
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229272/

[7229273] rados/standalone/{supported-random-distro$/{rhel_8} workloads/osd-backfill}
time:   03:26:00
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229273/

[7229274] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/redirect}
time:   00:13:22
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229274/

[7229275] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest}
time:   00:21:26
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229275/

[7229277] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_orch_cli}
time:   00:19:29
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229277/

[7229278] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:23:35
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229278/

[7229279] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:15:12
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229279/

[7229280] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/redirect_promote_tests}
time:   00:22:57
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229280/

[7229281] rados/singleton/{all/lost-unfound mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04}}
time:   00:31:10
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229281/

[7229282] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/redirect_set_object}
time:   00:13:58
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229282/

[7229283] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2}
time:   00:31:51
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229283/

[7229284] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites}
time:   00:21:37
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229284/

[7229285] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:25:12
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229285/

[7229288] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{ubuntu_20.04}}
time:   00:09:36
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229288/

[7229289] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/small-objects-balanced}
time:   00:21:20
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229289/

[7229290] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews}
time:   00:11:23
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229290/

[7229291] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados tasks/mon_recovery validater/lockdep}
time:   00:14:34
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229291/

[7229292] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04}}
time:   00:08:41
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229292/

[7229293] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/small-objects-localized}
time:   00:23:06
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229293/

[7229294] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add}
time:   00:16:23
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229294/

[7229296] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/small-objects}
time:   00:23:47
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229296/

[7229298] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/snaps-few-objects-balanced}
time:   00:27:28
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229298/

[7229299] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}}
time:   00:15:25
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229299/

[7229300] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/sync workloads/snaps-few-objects}
time:   00:31:15
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229300/

[7229301] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_orch_cli_mon}
time:   00:26:21
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229301/

[7229302] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/snaps-few-objects-localized}
time:   00:32:04
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229302/

[7229304] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects}
time:   00:33:00
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229304/

[7229306] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{rhel_8} tasks/crash}
time:   00:15:52
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229306/

[7229308] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{ubuntu_20.04}}
time:   00:09:30
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229308/

[7229309] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/write_fadvise_dontneed}
time:   00:21:07
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229309/

[7229310] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start}
time:   00:17:46
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229310/

[7229311] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04}}
time:   00:12:26
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229311/

[7229313] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-many-deletes}
time:   00:27:44
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229313/

[7229314] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_big}
time:   00:32:46
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229314/

[7229315] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache-agent-big}
time:   00:25:40
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229315/

[7229316] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/cache-snaps}
time:   00:32:52
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229316/

[7229317] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools} 1-start 2-services/rgw 3-final}
time:   00:15:16
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229317/

[7229318] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}}
time:   00:10:16
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229318/

[7229319] rados/standalone/{supported-random-distro$/{centos_8} workloads/osd}
time:   02:56:40
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229319/

[7229322] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/cache-agent-small}
time:   00:23:14
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229322/

[7229323] rados/singleton/{all/mon-auth-caps mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}}
time:   00:14:00
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229323/

[7229324] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy}
time:   00:26:27
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229324/

[7229327] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_adoption}
time:   00:13:46
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229327/

[7229328] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{centos_8}}
time:   00:15:33
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229328/

[7229329] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/cache-snaps-balanced}
time:   00:22:13
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229329/

[7229331] rados/singleton/{all/mon-config-key-caps mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04}}
time:   00:11:01
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229331/

[7229333] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados tasks/rados_api_tests validater/valgrind}
time:   00:54:19
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229333/

[7229334] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache-snaps}
time:   00:24:24
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229334/

[7229335] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-flag}
time:   00:15:31
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229335/

[7229336] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache}
time:   00:16:50
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229336/

[7229337] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-small-objects-overwrites}
time:   00:21:52
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229337/

[7229339] rados/singleton/{all/mon-config-keys mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}}
time:   00:18:35
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229339/

[7229340] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04}}
time:   00:32:27
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229340/

[7229343] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_cephadm}
time:   00:20:19
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229343/

[7229344] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/pool-create-delete}
time:   00:19:11
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229344/

[7229345] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04} tasks/rados_workunit_loadgen_mix}
time:   00:21:28
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229345/

[7229346] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/pool-snaps-few-objects}
time:   00:31:54
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229346/

[7229347] rados/singleton/{all/mon-config mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}}
time:   00:14:43
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229347/

[7229348] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-hybrid} supported-random-distro$/{centos_8} tasks/failover}
time:   00:16:30
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229348/

[7229349] rados/objectstore/{backends/objectstore-memstore supported-random-distro$/{centos_8}}
time:   00:11:57
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229349/

[7229350] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/rados_api_tests}
time:   00:33:38
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229350/

[7229351] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{ubuntu_20.04}}
time:   00:22:10
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229351/

[7229352] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start}
time:   00:16:55
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229352/

[7229353] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} thrashers/none thrashosds-health workloads/radosbench-high-concurrency}
time:   00:45:36
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229353/

[7229354] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects}
time:   00:24:56
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229354/

[7229355] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/radosbench}
time:   00:52:36
info:   http://pulpito.ceph.com/yuriw-2023-03-31_20:17:59-rados-reef-distro-default-smithi/7229355/