2023-04-17T14:27:26.548 INFO:root:teuthology version: 0.0.1.dev96+g8d156ae
2023-04-17T14:27:26.549 INFO:teuthology.results:subset: '111/12000'
2023-04-17T14:27:26.549 INFO:teuthology.results:seed: '9076'
2023-04-17T19:43:03.917 INFO:root:teuthology version: 0.0.1.dev96+g8d156ae
2023-04-17T19:43:03.943 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-04-17T22:21:20.223 INFO:teuthology.results:Tests finished! gathering results...
2023-04-17T22:21:53.176 INFO:scrape:Found 387 jobs
2023-04-17T22:22:02.702 INFO:scrape:Found 6 distinct failure reasons
2023-04-17T22:22:02.703 INFO:scrape:Failure: {'smithi103.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org/metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP: 152.19.134.198)", 'rc': 1, 'results': []}}
2023-04-17T22:22:02.704 INFO:scrape:31 jobs
2023-04-17T22:22:02.704 INFO:scrape:suites intersection: []
2023-04-17T22:22:02.705 INFO:scrape:suites union: ['0-nvme-loop', '0-start', '1-pg-log-overrides/normal_pg_log', '1-pg-log-overrides/short_pg_log', '1-start', '1-tasks', '2-ops/rm-zap-add}', '2-ops/rm-zap-flag}', '2-recovery-overrides/{more-async-partial-recovery}', '2-recovery-overrides/{more-async-recovery}', '2-recovery-overrides/{more-partial-recovery}', '2-services/basic', '3-final}', '3-scrub-overrides/{default}', '3-scrub-overrides/{max-simultaneous-scrubs-2}', '3-scrub-overrides/{max-simultaneous-scrubs-3}', 'agent/off', 'agent/on', 'backoff/normal', 'backoff/peering', 'backoff/peering_and_degraded', 'ceph', 'cluster/{12-osds', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'crc-failures/default', 'd-balancer/crush-compat', 'd-balancer/on', 'fast/fast', 'fixed-2', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/many', 'msgr-failures/osd-dispatch-delay', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'no_pools', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-low-osd-mem-target', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/basic/{ceph', 'rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0', 'rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8', 'rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0}', 'rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0', 'rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8', 'rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0', 'rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8', 'rados/multimon/{clusters/6', 'rados/objectstore/{backends/objectstore-bluestore-b', 'rados/objectstore/{backends/objectstore-memstore', 'rados/singleton-nomsgr/{all/admin_socket_output', 'rados/singleton-nomsgr/{all/balancer', 'rados/singleton-nomsgr/{all/crushdiff', 'rados/singleton-nomsgr/{all/large-omap-object-warnings', 'rados/singleton-nomsgr/{all/lazy_omap_stats_output', 'rados/singleton-nomsgr/{all/librados_hello_world', 'rados/singleton-nomsgr/{all/version-number-sanity', 'rados/singleton/{all/ec-inconsistent-hinfo', 'rados/singleton/{all/mon-config', 'rados/standalone/{supported-random-distro$/{rhel_8}', 'rados/thrash-erasure-code-big/{ceph', 'rados/thrash-erasure-code/{ceph', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'rados/upgrade/parallel/{0-random-distro$/{rhel_8.6_container_tools_3.0}', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'recovery-overrides/{more-async-recovery}', 'recovery-overrides/{more-partial-recovery}', 'start}', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{rhel_8}}', 'task/test_cephadm}', 'task/test_nfs}', 'task/test_orch_cli_mon}', 'tasks/mon_recovery}', 'tasks/rados_python}', 'test_rbd_api', 'test_rbd_python}}', 'thrashers/careful', 'thrashers/minsize_recovery', 'thrashers/morepggrow', 'thrashers/none', 'thrashers/pggrow', 'thrashosds-health', 'upgrade-sequence', 'workload/{ec-rados-default', 'workloads/admin_socket_objecter_requests}', 'workloads/cache-agent-small}', 'workloads/cache-snaps}', 'workloads/cache}', 'workloads/ec-rados-plugin=jerasure-k=4-m=2}', 'workloads/ec-small-objects-many-deletes}', 'workloads/osd-backfill}', 'workloads/small-objects}']
2023-04-17T22:22:02.705 INFO:scrape:
2023-04-17T22:22:02.705 INFO:scrape:Failure: Command failed on smithi172 with status 1: 'sudo yum install -y kernel'
2023-04-17T22:22:02.706 INFO:scrape:124 jobs
2023-04-17T22:22:02.707 INFO:scrape:suites intersection: []
2023-04-17T22:22:02.708 INFO:scrape:suites union: ['0-kubeadm', '0-nvme-loop', '0-size-min-size-overrides/2-size-2-min-size', '0-size-min-size-overrides/3-size-2-min-size', '1-install/nautilus-v1only', '1-install/pacific', '1-install/quincy', '1-pg-log-overrides/normal_pg_log', '1-pg-log-overrides/short_pg_log', '1-rook', '1-start', '2-inject-leak/mon', '2-inject-leak/none', '2-ops/repave-all}', '2-ops/rm-zap-add}', '2-ops/rm-zap-flag}', '2-ops/rm-zap-wait}', '2-ops/rmdir-reactivate}', '2-recovery-overrides/{default}', '2-recovery-overrides/{more-active-recovery}', '2-recovery-overrides/{more-async-partial-recovery}', '2-recovery-overrides/{more-async-recovery}', '2-recovery-overrides/{more-partial-recovery}', '2-services/basic', '2-services/rgw', '2-workload/none', '2-workload/radosbench', '3-final}', '3-scrub-overrides/{default}', '3-scrub-overrides/{max-simultaneous-scrubs-2}', '3-scrub-overrides/{max-simultaneous-scrubs-3}', 'agent/off', 'agent/on', 'backoff/normal', 'backoff/peering', 'backoff/peering_and_degraded', 'centos_latest}', 'ceph', 'cluster/1-node', 'cluster/3-node', 'cluster/{12-osds', 'clusters/3-mons', 'clusters/9-mons', 'clusters/{fixed-2', 'clusters/{fixed-4', 'clusters/{openstack', 'crc-failures/bad_map_crc_failure', 'crc-failures/default', 'd-balancer/crush-compat', 'd-balancer/on', 'd-thrash/default/{default', 'd-thrash/none', 'debug/mgr', 'fast/fast', 'fast/normal', 'fixed-2', 'k8s/1.21', 'mgr_ttl_cache/disable', 'mgr_ttl_cache/enable', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/many', 'msgr-failures/mon-delay', 'msgr-failures/none', 'msgr-failures/osd-delay', 'msgr-failures/osd-dispatch-delay', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'net/calico', 'net/flannel', 'net/host', 'no_pools', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-low-osd-mem-target', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/basic/{ceph', 'rados/cephadm/osds/{0-distro/centos_8.stream_container_tools', 'rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun', 'rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0', 'rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8', 'rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun}', 'rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools}', 'rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools', 'rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun', 'rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0', 'rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8', 'rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun', 'rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0', 'rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8', 'rados/dashboard/{0-single-container-host', 'rados/mgr/{clusters/{2-node-mgr}', 'rados/monthrash/{ceph', 'rados/multimon/{clusters/21', 'rados/multimon/{clusters/3', 'rados/objectstore/{backends/ceph_objectstore_tool', 'rados/objectstore/{backends/fusestore', 'rados/objectstore/{backends/keyvaluedb', 'rados/rest/{mgr-restful', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rados/singleton-nomsgr/{all/balancer', 'rados/singleton-nomsgr/{all/cache-fs-trunc', 'rados/singleton-nomsgr/{all/crushdiff', 'rados/singleton-nomsgr/{all/large-omap-object-warnings', 'rados/singleton-nomsgr/{all/librados_hello_world', 'rados/singleton-nomsgr/{all/osd_stale_reads', 'rados/singleton-nomsgr/{all/pool-access', 'rados/singleton-nomsgr/{all/recovery-unfound-found', 'rados/singleton/{all/mon-config', 'rados/singleton/{all/osd-recovery-incomplete', 'rados/singleton/{all/peer', 'rados/singleton/{all/pg-autoscaler', 'rados/singleton/{all/pg-removal-interruption', 'rados/singleton/{all/rebuild-mondb', 'rados/singleton/{all/recovery-preemption', 'rados/singleton/{all/resolve_stuck_peering', 'rados/singleton/{all/test-crash', 'rados/singleton/{all/test-noautoscale-flag', 'rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/rhel_latest', 'rados/singleton/{all/thrash-backfill-full', 'rados/singleton/{all/thrash-eio', 'rados/singleton/{all/thrash-rados/{thrash-rados', 'rados/standalone/{supported-random-distro$/{centos_8}', 'rados/standalone/{supported-random-distro$/{rhel_8}', 'rados/thrash-erasure-code-big/{ceph', 'rados/thrash-erasure-code-isa/{arch/x86_64', 'rados/thrash-erasure-code-overwrites/{bluestore-bitmap', 'rados/thrash-erasure-code-shec/{ceph', 'rados/thrash-erasure-code/{ceph', 'rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools}', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'rados/valgrind-leaks/{1-start', 'rados/verify/{centos_latest', 'random-objectstore$/{bluestore-bitmap}', 'random-objectstore$/{bluestore-comp-snappy}', 'random-objectstore$/{bluestore-comp-zlib}', 'random-objectstore$/{bluestore-comp-zstd}', 'recovery-overrides/{default}', 'recovery-overrides/{more-active-recovery}', 'recovery-overrides/{more-async-partial-recovery}', 'recovery-overrides/{more-async-recovery}', 'recovery-overrides/{more-partial-recovery}', 'rook/1.7.2}', 'rook/master}', 'start}', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{centos_8}}', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{rhel_8}}', 'task/test_adoption}', 'task/test_cephadm}', 'task/test_iscsi_pids_limit/{centos_8.stream_container_tools', 'task/test_nfs}', 'task/test_orch_cli_mon}', 'task/test_orch_cli}', 'tasks/crash}', 'tasks/dashboard}', 'tasks/e2e}', 'tasks/failover}', 'tasks/insights}', 'tasks/mon_clock_no_skews}', 'tasks/mon_recovery', 'tasks/mon_recovery}', 'tasks/rados_api_tests', 'tasks/rados_cls_all', 'tasks/rados_workunit_loadgen_mix}', 'tasks/readwrite}', 'tasks/scrub_test}', 'tasks/workunits}', 'test_envlibrados_for_rocksdb}', 'test_iscsi_pids_limit}}', 'thrashers/careful', 'thrashers/default', 'thrashers/mapgap', 'thrashers/morepggrow', 'thrashers/none', 'thrashers/one', 'thrashers/pggrow', 'thrashers/sync-many', 'thrashosds-health', 'thrashosds-health}', 'three-plus-one}', 'validater/lockdep}', 'validater/valgrind}', 'workloads/cache-agent-big}', 'workloads/cache-agent-small}', 'workloads/cache-pool-snaps-readproxy}', 'workloads/cache-pool-snaps}', 'workloads/cache-snaps-balanced}', 'workloads/cache-snaps}', 'workloads/crush}', 'workloads/dedup-io-mixed}', 'workloads/dedup-io-snaps}', 'workloads/ec-pool-snaps-few-objects-overwrites}', 'workloads/ec-rados-plugin=isa-k=2-m=1}', 'workloads/ec-rados-plugin=jerasure-k=2-m=1}', 'workloads/ec-rados-plugin=jerasure-k=4-m=2}', 'workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}', 'workloads/ec-rados-plugin=shec-k=4-m=3-c=2}', 'workloads/misc}', 'workloads/pool-snaps-few-objects}', 'workloads/rados_api_tests}', 'workloads/rados_mon_workunits}', 'workloads/radosbench-high-concurrency}', 'workloads/radosbench}', 'workloads/rbd_cls}', 'workloads/redirect_promote_tests}', 'workloads/redirect_set_object}', 'workloads/redirect}', 'workloads/set-chunks-read}', 'workloads/small-objects}', 'workloads/snaps-few-objects-balanced}', 'workloads/snaps-few-objects-localized}', 'workloads/snaps-few-objects}', 'workloads/test_rbd_api}', 'workloads/write_fadvise_dontneed}']
2023-04-17T22:22:02.708 INFO:scrape:
2023-04-17T22:22:02.708 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi027 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2c4c55d1dca0ff47b0c5a7658cede7f55c32237d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh'
2023-04-17T22:22:02.709 INFO:scrape:1 jobs: ['7244397']
2023-04-17T22:22:02.709 INFO:scrape:suites: ['debug/mgr', 'mon_election/classic', 'rados/dashboard/{0-single-container-host', 'random-objectstore$/{bluestore-comp-lz4}', 'tasks/e2e}']
2023-04-17T22:22:02.709 INFO:scrape:
2023-04-17T22:22:02.710 INFO:scrape:Failure: Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi002 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2c4c55d1dca0ff47b0c5a7658cede7f55c32237d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh'
2023-04-17T22:22:02.710 INFO:scrape:1 jobs: ['7244211']
2023-04-17T22:22:02.710 INFO:scrape:suites: ['mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v2only', 'objectstore/bluestore-low-osd-mem-target', 'rados', 'rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/centos_latest', 'supported-random-distro$/{ubuntu_latest}}', 'test_envlibrados_for_rocksdb}']
2023-04-17T22:22:02.710 INFO:scrape:
2023-04-17T22:22:02.711 INFO:scrape:Failure: "1681762067.1775582 osd.0 (osd.0) 24 : cluster [WRN] WaitReplicas::react(const DigestUpdate&): Unexpected DigestUpdate event" in cluster log
2023-04-17T22:22:02.711 INFO:scrape:2 jobs: ['7244443', '7244072']
2023-04-17T22:22:02.711 INFO:scrape:suites intersection: ['rados', 'supported-random-distro$/{ubuntu_latest}}']
2023-04-17T22:22:02.712 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'msgr-failures/many', 'msgr/async', 'objectstore/bluestore-bitmap', 'rados', 'rados/singleton-nomsgr/{all/multi-backfill-reject', 'rados/singleton/{all/osd-backfill', 'supported-random-distro$/{ubuntu_latest}}']
2023-04-17T22:22:02.712 INFO:scrape:
2023-04-17T22:22:02.712 INFO:scrape:Failure: {'smithi167.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org/metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP: 38.145.60.20)", 'rc': 1, 'results': []}, 'smithi158.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org/metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP: 152.19.134.142)", 'rc': 1, 'results': []}, 'smithi019.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org/metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP: 67.219.144.68)", 'rc': 1, 'results': []}, 'smithi012.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file': None, 'disable_excludes': None, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_dir': None, 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot': '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'], 'releasever': None, 'security': False, 'skip_broken': False, 'state': 'present', 'update_cache': False, 'update_only': False, 'validate_certs': True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org/metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP: 67.219.144.68)", 'rc': 1, 'results': []}}
2023-04-17T22:22:02.712 INFO:scrape:1 jobs: ['7244371']
2023-04-17T22:22:02.713 INFO:scrape:suites: ['clusters/{fixed-4', 'mon_election/connectivity', 'msgr-failures/osd-dispatch-delay', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/thrash-erasure-code-shec/{ceph', 'recovery-overrides/{more-active-recovery}', 'supported-random-distro$/{rhel_8}', 'thrashers/default', 'thrashosds-health', 'workloads/ec-rados-plugin=shec-k=4-m=3-c=2}']
2023-04-17T22:22:02.713 INFO:scrape:
2023-04-17T22:22:02.731 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/
failed:  128
dead:    32
running: 0
waiting: 0
queued:  0
passed:  227


Fail
=================================================================
[7244058]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/snaps-few-objects}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244058/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244058/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f7f4d49e5c2741fcb675aa24dd46b230

    Command failed on smithi157 with status 1: 'sudo yum install -y kernel'

[7244060]  rados/singleton/{all/mon-config mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244060/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244060/
sentry: https://sentry.ceph.com/organizations/ceph/?query=90472242a33d446fba8c9b6300e05637

    Command failed on smithi049 with status 1: 'sudo yum install -y kernel'

[7244061]  rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244061/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244061/
sentry: https://sentry.ceph.com/organizations/ceph/?query=96ce431f64dc43bd9d715bb423c21eaa

    Command failed on smithi144 with status 1: 'sudo yum install -y kernel'

[7244063]  rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/classic start}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244063/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244063/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e07cde7b4a9448749fb55c374ac410c8

    Command failed on smithi176 with status 1: 'sudo yum install -y kernel'

[7244064]  rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{rhel_8} tasks/workunits}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244064/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244064/
sentry: https://sentry.ceph.com/organizations/ceph/?query=61edf96116c34149966bf62c10a5b7e9

    Command failed on smithi087 with status 1: 'sudo yum install -y kernel'

[7244069]  rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244069/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244069/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5718ed1199074e159faf4baff8412fa1

    Command failed on smithi077 with status 1: 'sudo yum install -y kernel'

[7244071]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-agent-small}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244071/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244071/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5cda1ed8637c4efbad67b39c244991c5

    Command failed on smithi203 with status 1: 'sudo yum install -y kernel'

[7244072]  rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:38:21
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244072/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244072/

    "1681744560.1670651 osd.2 (osd.2) 6 : cluster [WRN]
    WaitReplicas::react(const DigestUpdate&): Unexpected DigestUpdate event" in
    cluster log

[7244074]  rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244074/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244074/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2823293439fd470ca443ff4f1ceee270

    Command failed on smithi003 with status 1: 'sudo yum install -y kernel'

[7244075]  rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244075/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244075/
sentry: https://sentry.ceph.com/organizations/ceph/?query=69d923c3d6284411acb4af8a997a6787

    Command failed on smithi116 with status 1: 'sudo yum install -y kernel'

[7244076]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-pool-snaps-readproxy}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244076/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244076/
sentry: https://sentry.ceph.com/organizations/ceph/?query=309b95ba8f02417a9383424a01aa3fbf

    Command failed on smithi156 with status 1: 'sudo yum install -y kernel'

[7244077]  rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} tasks/mon_recovery}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244077/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244077/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e591994ca65c4f0ba9731770ba0f960b

    Command failed on smithi060 with status 1: 'sudo yum install -y kernel'

[7244078]  rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244078/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244078/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4c886c89c64a418dab461fbf43f12d4f

    Command failed on smithi090 with status 1: 'sudo yum install -y kernel'

[7244079]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-pool-snaps}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244079/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244079/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e86893019930418bb0564b27c12ba324

    Command failed on smithi066 with status 1: 'sudo yum install -y kernel'

[7244080]  rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_mix}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244080/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244080/
sentry: https://sentry.ceph.com/organizations/ceph/?query=27dbdb7c53554db3b11a66f4a5f6562e

    Command failed on smithi153 with status 1: 'sudo yum install -y kernel'

[7244081]  rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_adoption}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244081/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244081/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7268cbe539244392bf01b482883442fb

    Command failed on smithi012 with status 1: 'sudo yum install -y kernel'

[7244082]  rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244082/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244082/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d8ee54e6705e417684737b76e6286698

    Command failed on smithi142 with status 1: 'sudo yum install -y kernel'

[7244084]  rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244084/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244084/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e3c2e475e7654e3cb6fe9cc465ef71d2

    Command failed on smithi121 with status 1: 'sudo yum install -y kernel'

[7244085]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/cache-snaps}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244085/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244085/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f305ad7fc5d740caa586c5c5b11babae

    Command failed on smithi186 with status 1: 'sudo yum install -y kernel'

[7244089]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/dedup-io-mixed}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244089/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244089/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9a4f4c3cf11f49d48da139fc34675a6c

    Command failed on smithi150 with status 1: 'sudo yum install -y kernel'

[7244090]  rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/one workloads/rados_mon_workunits}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244090/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244090/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d91d12738a8b400eaaed291912ed60e8

    Command failed on smithi072 with status 1: 'sudo yum install -y kernel'

[7244092]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/dedup-io-snaps}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244092/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244092/
sentry: https://sentry.ceph.com/organizations/ceph/?query=cf56f0cd3e1c4a4a8d9a0713de118aa8

    Command failed on smithi187 with status 1: 'sudo yum install -y kernel'

[7244094]  rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244094/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244094/
sentry: https://sentry.ceph.com/organizations/ceph/?query=716df700982448579a50b79827f68840

    Command failed on smithi078 with status 1: 'sudo yum install -y kernel'

[7244095]  rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/pacific backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/test_rbd_api}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244095/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244095/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9409302f3e144f928908b5f32c450a65

    Command failed on smithi157 with status 1: 'sudo yum install -y kernel'

[7244097]  rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244097/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244097/
sentry: https://sentry.ceph.com/organizations/ceph/?query=6517d87ad6b24c43992a0366a34911b5

    Command failed on smithi130 with status 1: 'sudo yum install -y kernel'

[7244098]  rados/singleton/{all/peer mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244098/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244098/
sentry: https://sentry.ceph.com/organizations/ceph/?query=78d02663aa8d496db0ff32b3671b1d2a

    Command failed on smithi182 with status 1: 'sudo yum install -y kernel'

[7244099]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/rados_api_tests}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244099/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244099/
sentry: https://sentry.ceph.com/organizations/ceph/?query=6a9b80c19fbd4f79bd0363944e43c867

    Command failed on smithi047 with status 1: 'sudo yum install -y kernel'

[7244101]  rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_cephadm}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244101/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244101/
sentry: https://sentry.ceph.com/organizations/ceph/?query=69cf7ebc3dae45a59d223d7ea1a30d77

    Command failed on smithi158 with status 1: 'sudo yum install -y kernel'

[7244102]  rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-bitmap} tasks/dashboard}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244102/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244102/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8b55f07b08e3432ca9e84fd00c90b28b

    Command failed on smithi181 with status 1: 'sudo yum install -y kernel'

[7244103]  rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{centos_8} tasks/crash}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244103/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244103/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ee7bce03a581448cad48b8585d052e81

    Command failed on smithi042 with status 1: 'sudo yum install -y kernel'

[7244104]  rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244104/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244104/
sentry: https://sentry.ceph.com/organizations/ceph/?query=29f7ac8f97c941029f155f73e5e2354c

    Command failed on smithi143 with status 1: 'sudo yum install -y kernel'

[7244105]  rados/rest/{mgr-restful supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244105/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244105/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c19cf22c1a254db1a6a544b9f42cc72c

    Command failed on smithi077 with status 1: 'sudo yum install -y kernel'

[7244106]  rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/calico rook/1.7.2}
-----------------------------------------------------------------
time:   00:06:23
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244106/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244106/
sentry: https://sentry.ceph.com/organizations/ceph/?query=522e432c24134c71955247d165a2035b

    Command failed on smithi195 with status 1: 'sudo systemctl enable --now
    kubelet && sudo kubeadm config images pull'

[7244110]  rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244110/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244110/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a5bdcf3be0454e50b9a35e4f8f4b9a17

    Command failed on smithi100 with status 1: 'sudo yum install -y kernel'

[7244115]  rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244115/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244115/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5fc5b437c24548fb84e4d440507e51a6

    Command failed on smithi066 with status 1: 'sudo yum install -y kernel'

[7244116]  rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244116/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244116/
sentry: https://sentry.ceph.com/organizations/ceph/?query=19c575ad6d2845a6ae84a2fa3ac99313

    Command failed on smithi121 with status 1: 'sudo yum install -y kernel'

[7244117]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/redirect}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244117/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244117/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ef861e28d62341959a89314eca336c54

    Command failed on smithi023 with status 1: 'sudo yum install -y kernel'

[7244118]  rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244118/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244118/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4a3d937900b448df96dddfa34996bd4e

    Command failed on smithi179 with status 1: 'sudo yum install -y kernel'

[7244119]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/redirect_promote_tests}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244119/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244119/
sentry: https://sentry.ceph.com/organizations/ceph/?query=43f11b4233cd483ba26ecffaef9f5852

    Command failed on smithi047 with status 1: 'sudo yum install -y kernel'

[7244121]  rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244121/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244121/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7fe387b758274640bcfb0cb3385e265c

    Command failed on smithi175 with status 1: 'sudo yum install -y kernel'

[7244122]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/redirect_set_object}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244122/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244122/
sentry: https://sentry.ceph.com/organizations/ceph/?query=59fe68af1f774dee8c3529817bbe4746

    Command failed on smithi038 with status 1: 'sudo yum install -y kernel'

[7244123]  rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244123/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244123/
sentry: https://sentry.ceph.com/organizations/ceph/?query=94d81b9d239b46898d9cb1f6643385ee

    Command failed on smithi115 with status 1: 'sudo yum install -y kernel'

[7244124]  rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244124/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244124/
sentry: https://sentry.ceph.com/organizations/ceph/?query=dae8b172f5d34b9d9595f69769de5d0f

    Command failed on smithi144 with status 1: 'sudo yum install -y kernel'

[7244125]  rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244125/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244125/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8b2759908a874b7d854cc07b18dbd2fc

    Command failed on smithi033 with status 1: 'sudo yum install -y kernel'

[7244126]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/set-chunks-read}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244126/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244126/
sentry: https://sentry.ceph.com/organizations/ceph/?query=621900edef60452bbd68d4ba14278f72

    Command failed on smithi066 with status 1: 'sudo yum install -y kernel'

[7244127]  rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools} 1-start 2-services/basic 3-final}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244127/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244127/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f002ff974b674e8a8afafe404342dac0

    Command failed on smithi186 with status 1: 'sudo yum install -y kernel'

[7244130]  rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244130/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244130/
sentry: https://sentry.ceph.com/organizations/ceph/?query=82e2ccdcf9e84cca8980d4a4faa66736

    Command failed on smithi019 with status 1: 'sudo yum install -y kernel'

[7244132]  rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244132/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244132/
sentry: https://sentry.ceph.com/organizations/ceph/?query=cccbe55600214e079b7d70f6dd0e0244

    Command failed on smithi050 with status 1: 'sudo yum install -y kernel'

[7244134]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/small-objects}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244134/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244134/
sentry: https://sentry.ceph.com/organizations/ceph/?query=70568612c9394176878a88bd253a9f26

    Command failed on smithi043 with status 1: 'sudo yum install -y kernel'

[7244135]  rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/snaps-few-objects}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244135/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244135/
sentry: https://sentry.ceph.com/organizations/ceph/?query=13651287c7c94ac3a98b7a2798b2ccea

    Command failed on smithi047 with status 1: 'sudo yum install -y kernel'

[7244136]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-balanced}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244136/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244136/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1489a3665c2741d6b411a009b35f8ccb

    Command failed on smithi155 with status 1: 'sudo yum install -y kernel'

[7244139]  rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244139/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244139/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b0d4028f7f5b492db5963f379387b515

    Command failed on smithi188 with status 1: 'sudo yum install -y kernel'

[7244140]  rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/readwrite}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244140/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244140/
sentry: https://sentry.ceph.com/organizations/ceph/?query=724f30fecc0f4aad9a08e46ba1e9d51a

    Command failed on smithi153 with status 1: 'sudo yum install -y kernel'

[7244142]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/snaps-few-objects}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244142/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244142/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8a2a20a0394e42efb2f10488dcd074ba

    Command failed on smithi002 with status 1: 'sudo yum install -y kernel'

[7244144]  rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244144/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244144/
sentry: https://sentry.ceph.com/organizations/ceph/?query=86cae944703442c09e3341182902cdcc

    Command failed on smithi078 with status 1: 'sudo yum install -y kernel'

[7244145]  rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{rhel_8} tasks/failover}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244145/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244145/
sentry: https://sentry.ceph.com/organizations/ceph/?query=885d8cbd81da462e9abb9f58b88e5483

    Command failed on smithi203 with status 1: 'sudo yum install -y kernel'

[7244149]  rados/standalone/{supported-random-distro$/{centos_8} workloads/crush}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244149/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244149/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4da89247b2fe47a1881ab02572fd7569

    Command failed on smithi117 with status 1: 'sudo yum install -y kernel'

[7244152]  rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/cache-snaps}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244152/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244152/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b87af752c7424caa9c06915ef11ff1a3

    Command failed on smithi114 with status 1: 'sudo yum install -y kernel'

[7244154]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache-agent-big}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244154/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244154/
sentry: https://sentry.ceph.com/organizations/ceph/?query=62158c11e2394e4590169a92ab7d5cce

    Command failed on smithi099 with status 1: 'sudo yum install -y kernel'

[7244158]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-agent-small}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244158/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244158/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9895656648f44ee5a3a4c1d8383df3d4

    Command failed on smithi077 with status 1: 'sudo yum install -y kernel'

[7244159]  rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_iscsi_pids_limit/{centos_8.stream_container_tools test_iscsi_pids_limit}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244159/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244159/
sentry: https://sentry.ceph.com/organizations/ceph/?query=19182fc262a74c84b72491e08fc1ee87

    Command failed on smithi017 with status 1: 'sudo yum install -y kernel'

[7244161]  rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244161/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244161/
sentry: https://sentry.ceph.com/organizations/ceph/?query=41478e2edcb3471088fefc67d82137c3

    Command failed on smithi002 with status 1: 'sudo yum install -y kernel'

[7244162]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache-pool-snaps-readproxy}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244162/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244162/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8eabc97df61c42e98399201621eaea2d

    Command failed on smithi063 with status 1: 'sudo yum install -y kernel'

[7244163]  rados/singleton-nomsgr/{all/crushdiff mon_election/connectivity rados supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244163/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244163/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e6eee900c458418fb71d9d5bace3d959

    Command failed on smithi097 with status 1: 'sudo yum install -y kernel'

[7244164]  rados/objectstore/{backends/fusestore supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244164/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244164/
sentry: https://sentry.ceph.com/organizations/ceph/?query=aa90b755b07b4050b75b88120449691d

    Command failed on smithi174 with status 1: 'sudo yum install -y kernel'

[7244165]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244165/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244165/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e19ba0e078014e56a2a993605ab105d5

    Command failed on smithi121 with status 1: 'sudo yum install -y kernel'

[7244167]  rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244167/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244167/
sentry: https://sentry.ceph.com/organizations/ceph/?query=abd711d9928e457eb395ce298642ab4a

    Command failed on smithi115 with status 1: 'sudo yum install -y kernel'

[7244168]  rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244168/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244168/
sentry: https://sentry.ceph.com/organizations/ceph/?query=654014fc40ad4196bc8eef3c73588c4b

    Command failed on smithi179 with status 1: 'sudo yum install -y kernel'

[7244169]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache-snaps-balanced}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244169/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244169/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d0659d56324d425abc7333e88bb062ee

    Command failed on smithi023 with status 1: 'sudo yum install -y kernel'

[7244170]  rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244170/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244170/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ca3d63c2c71946509ca9872192ee134f

    Command failed on smithi175 with status 1: 'sudo yum install -y kernel'

[7244172]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-snaps}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244172/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244172/
sentry: https://sentry.ceph.com/organizations/ceph/?query=fc17313354a0496ca2bc16bb4287140f

    Command failed on smithi151 with status 1: 'sudo yum install -y kernel'

[7244175]  rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_nfs}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244175/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244175/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1ad18d38bf054033970e2cfff0028943

    Command failed on smithi193 with status 1: 'sudo yum install -y kernel'

[7244176]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/dedup-io-mixed}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244176/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244176/
sentry: https://sentry.ceph.com/organizations/ceph/?query=023a85772e074500ac86877a60d3677f

    Command failed on smithi156 with status 1: 'sudo yum install -y kernel'

[7244179]  rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244179/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244179/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4e00c5068f50426dba8fa15ea032a0dd

    Command failed on smithi150 with status 1: 'sudo yum install -y kernel'

[7244180]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/dedup-io-snaps}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244180/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244180/
sentry: https://sentry.ceph.com/organizations/ceph/?query=622318fa152548f1879c3286ee5f6529

    Command failed on smithi103 with status 1: 'sudo yum install -y kernel'

[7244181]  rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244181/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244181/
sentry: https://sentry.ceph.com/organizations/ceph/?query=84484b3b811a4004975c4cbd618b6d16

    Command failed on smithi195 with status 1: 'sudo yum install -y kernel'

[7244184]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/rados_api_tests}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244184/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244184/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8c70ef177451467987e02b4b1b8f4f31

    Command failed on smithi179 with status 1: 'sudo yum install -y kernel'

[7244185]  rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244185/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244185/
sentry: https://sentry.ceph.com/organizations/ceph/?query=81cd3ca6c73f476a98f27078b87bba48

    Command failed on smithi090 with status 1: 'sudo yum install -y kernel'

[7244186]  rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-bitmap} supported-random-distro$/{rhel_8} tasks/insights}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244186/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244186/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5c9bfdf72e924830a3cde644c3d8a7ef

    Command failed on smithi064 with status 1: 'sudo yum install -y kernel'

[7244187]  rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244187/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244187/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9e922395f2734b9498e2da70e7a6fdf7

    Command failed on smithi060 with status 1: 'sudo yum install -y kernel'

[7244189]  rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244189/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244189/
sentry: https://sentry.ceph.com/organizations/ceph/?query=6288fbbe34b847e493269cdb9bc2a3bc

    Command failed on smithi153 with status 1: 'sudo yum install -y kernel'

[7244193]  rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244193/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244193/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d5bde9ef16dc4e4d823d094bb31aadcd

    Command failed on smithi112 with status 1: 'sudo yum install -y kernel'

[7244196]  rados/singleton/{all/test-noautoscale-flag mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244196/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244196/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9a762b8aef654b89b85482dab74c1754

    Command failed on smithi156 with status 1: 'sudo yum install -y kernel'

[7244197]  rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_orch_cli}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244197/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244197/
sentry: https://sentry.ceph.com/organizations/ceph/?query=06a28e6d1da84709b851cc8c4def4f6e

    Command failed on smithi050 with status 1: 'sudo yum install -y kernel'

[7244198]  rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} tasks/scrub_test}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244198/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244198/
sentry: https://sentry.ceph.com/organizations/ceph/?query=3c01101fcd494c8b9638ba493d522958

    Command failed on smithi047 with status 1: 'sudo yum install -y kernel'

[7244199]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/redirect_promote_tests}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244199/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244199/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a1a333ff49474f81b5936fb24a1a3df9

    Command failed on smithi181 with status 1: 'sudo yum install -y kernel'

[7244200]  rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/quincy backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/radosbench}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244200/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244200/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9b1227e09f9c49f2a09542e8a7e9267d

    Command failed on smithi114 with status 1: 'sudo yum install -y kernel'

[7244201]  rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244201/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244201/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a950248738934cd9bb23678cf4cd3862

    Command failed on smithi120 with status 1: 'sudo yum install -y kernel'

[7244202]  rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-bitmap} tasks/e2e}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244202/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244202/
sentry: https://sentry.ceph.com/organizations/ceph/?query=70e569d27cbb4b5caddd7f715b893954

    Command failed on smithi187 with status 1: 'sudo yum install -y kernel'

[7244203]  rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/flannel rook/master}
-----------------------------------------------------------------
time:   00:06:26
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244203/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244203/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7b21da2ec0224f329879db9c973726cd

    Command failed on smithi066 with status 1: 'sudo systemctl enable --now
    kubelet && sudo kubeadm config images pull'

[7244204]  rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244204/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244204/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9b0ab27839ba4c6ab8ffebc220881b26

    Command failed on smithi150 with status 1: 'sudo yum install -y kernel'

[7244205]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/redirect_set_object}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244205/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244205/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d42973e556e946a583e02edf5be6226d

    Command failed on smithi017 with status 1: 'sudo yum install -y kernel'

[7244207]  rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244207/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244207/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f51ea8fe32ff43ebadb22df633831604

    Command failed on smithi144 with status 1: 'sudo yum install -y kernel'

[7244210]  rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244210/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244210/
sentry: https://sentry.ceph.com/organizations/ceph/?query=34116a3f3c6c4ebd91e5c94113fc8bc2

    Command failed on smithi153 with status 1: 'sudo yum install -y kernel'

[7244211]  rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/centos_latest test_envlibrados_for_rocksdb} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:12:01
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244211/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244211/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2432d2067077435d831f627ceaaae98e

    Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on
    smithi002 with status 2: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=2c4c55d1dca0ff47b0c5a7658cede7f55c32237d
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh'

[7244216]  rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/rgw 3-final}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244216/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244216/
sentry: https://sentry.ceph.com/organizations/ceph/?query=dcb0778311554a58b2c9dec8b119b682

    Command failed on smithi012 with status 1: 'sudo yum install -y kernel'

[7244218]  rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244218/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244218/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d62e2bdad765493786d6127a8722cc9e

    Command failed on smithi107 with status 1: 'sudo yum install -y kernel'

[7244219]  rados/objectstore/{backends/keyvaluedb supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244219/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244219/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1ccacf8b069242d3ac153f8c70313c85

    Command failed on smithi115 with status 1: 'sudo yum install -y kernel'

[7244220]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/snaps-few-objects-balanced}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244220/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244220/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9a7b1b64bb1a48a28a1e1e3f13c276b0

    Command failed on smithi202 with status 1: 'sudo yum install -y kernel'

[7244222]  rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_orch_cli_mon}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244222/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244222/
sentry: https://sentry.ceph.com/organizations/ceph/?query=86da0a7d6a0749adafbb6d98535a2cd0

    Command failed on smithi131 with status 1: 'sudo yum install -y kernel'

[7244223]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/snaps-few-objects-localized}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244223/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244223/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e597a90532934c53ac085f704ac4b746

    Command failed on smithi186 with status 1: 'sudo yum install -y kernel'

[7244224]  rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244224/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244224/
sentry: https://sentry.ceph.com/organizations/ceph/?query=89d88bb3b9d04997a2b7f779e193b924

    Command failed on smithi019 with status 1: 'sudo yum install -y kernel'

[7244225]  rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244225/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244225/
sentry: https://sentry.ceph.com/organizations/ceph/?query=6231592d2b8a47dfb3cc609afe7be91e

    Command failed on smithi038 with status 1: 'sudo yum install -y kernel'

[7244228]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/write_fadvise_dontneed}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244228/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244228/
sentry: https://sentry.ceph.com/organizations/ceph/?query=3a459734707b4599bd8eee8d4399f65b

    Command failed on smithi017 with status 1: 'sudo yum install -y kernel'

[7244231]  rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244231/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244231/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7db73700f3bd40efaa37e1a895bed73d

    Command failed on smithi193 with status 1: 'sudo yum install -y kernel'

[7244236]  rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244236/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244236/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1595e7bddf50487599271b83c8b8efe8

    Command failed on smithi157 with status 1: 'sudo yum install -y kernel'

[7244237]  rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244237/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244237/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f3a1f1c03b1a477f9b47ba8f6459c8e0

    Command failed on smithi107 with status 1: 'sudo yum install -y kernel'

[7244239]  rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244239/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244239/
sentry: https://sentry.ceph.com/organizations/ceph/?query=eca057e0da3c44f4ae006fbe43ec60b2

    Command failed on smithi100 with status 1: 'sudo yum install -y kernel'

[7244245]  rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244245/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244245/
sentry: https://sentry.ceph.com/organizations/ceph/?query=cf41950bc2d9423ab689bb59780c82be

    Command failed on smithi172 with status 1: 'sudo yum install -y kernel'

[7244250]  rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244250/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244250/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e0e41cb3b0c84d1484c6535eed336abd

    Command failed on smithi088 with status 1: 'sudo yum install -y kernel'

[7244251]  rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244251/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244251/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b95e684682af454f82249f6de0035652

    Command failed on smithi155 with status 1: 'sudo yum install -y kernel'

[7244252]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-snaps-balanced}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244252/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244252/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2444e07056184aafbf4def9ff01f53e2

    Command failed on smithi107 with status 1: 'sudo yum install -y kernel'

[7244253]  rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/rbd_cls}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244253/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244253/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c4a7afb5bab54d4fb3ab3a38ce721423

    Command failed on smithi143 with status 1: 'sudo yum install -y kernel'

[7244254]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache-snaps}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244254/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244254/
sentry: https://sentry.ceph.com/organizations/ceph/?query=86c751994be34f05bef6be44506f15c4

    Command failed on smithi047 with status 1: 'sudo yum install -y kernel'

[7244265]  rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/rhel_latest test_envlibrados_for_rocksdb} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244265/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244265/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b10acbc34b0b408fa62d7df8aeeff718

    Command failed on smithi182 with status 1: 'sudo yum install -y kernel'

[7244267]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/pool-snaps-few-objects}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244267/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244267/
sentry: https://sentry.ceph.com/organizations/ceph/?query=6f95d0fa54ad4a12b2a5de95d58ac77c

    Command failed on smithi144 with status 1: 'sudo yum install -y kernel'

[7244268]  rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244268/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244268/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5a7eb79ab7b34160a265534f1fe6ce63

    Command failed on smithi038 with status 1: 'sudo yum install -y kernel'

[7244273]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/radosbench-high-concurrency}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244273/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244273/
sentry: https://sentry.ceph.com/organizations/ceph/?query=6ee0faa8018c400f849945ae3af6581c

    Command failed on smithi187 with status 1: 'sudo yum install -y kernel'

[7244278]  rados/standalone/{supported-random-distro$/{rhel_8} workloads/misc}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244278/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244278/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9c10c16419d7451ebb8df8a39a300bb4

    Command failed on smithi078 with status 1: 'sudo yum install -y kernel'

[7244279]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/redirect}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244279/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244279/
sentry: https://sentry.ceph.com/organizations/ceph/?query=15fbda34d9ef41f890e50c31c1c243d6

    Command failed on smithi019 with status 1: 'sudo yum install -y kernel'

[7244280]  rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244280/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244280/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5492357070ec4aa0bd0493f70b493d12

    Command failed on smithi121 with status 1: 'sudo yum install -y kernel'

[7244281]  rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244281/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244281/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d45329d494d24f4baa494ba47b503312

    Command failed on smithi148 with status 1: 'sudo yum install -y kernel'

[7244283]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/redirect_promote_tests}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244283/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244283/
sentry: https://sentry.ceph.com/organizations/ceph/?query=64af99fa2a244bbb970177cbe365fbe6

    Command failed on smithi182 with status 1: 'sudo yum install -y kernel'

[7244299]  rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/host rook/1.7.2}
-----------------------------------------------------------------
time:   00:06:16
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244299/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244299/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e29de473714d4f4d86f59b2bbc02a2e4

    Command failed on smithi121 with status 1: 'sudo systemctl enable --now
    kubelet && sudo kubeadm config images pull'

[7244315]  rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_iscsi_pids_limit/{centos_8.stream_container_tools test_iscsi_pids_limit}}
-----------------------------------------------------------------
time:   00:38:30
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244315/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244315/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7e53f60a72344104807a550419816fe6

    Command failed on smithi107 with status 1: 'TESTDIR=/home/ubuntu/cephtest
    bash -s'

[7244397]  rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-lz4} tasks/e2e}
-----------------------------------------------------------------
time:   02:05:28
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244397/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244397/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4be0cb9e60a44710ba1d327a6bb174a1

    Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi027
    with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd
    -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=2c4c55d1dca0ff47b0c5a7658cede7f55c32237d
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh'

[7244399]  rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/calico rook/master}
-----------------------------------------------------------------
time:   00:06:18
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244399/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244399/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c1be9c207ccb4183bba6eb0bab4aff11

    Command failed on smithi158 with status 1: 'sudo systemctl enable --now
    kubelet && sudo kubeadm config images pull'

[7244443]  rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:24:05
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244443/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244443/

    "1681762067.1775582 osd.0 (osd.0) 24 : cluster [WRN]
    WaitReplicas::react(const DigestUpdate&): Unexpected DigestUpdate event" in
    cluster log



Dead
=================================================================
[7244286]  rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2}
-----------------------------------------------------------------
time:   00:38:33
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244286/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244286/
sentry: https://sentry.ceph.com/organizations/ceph/?query=821ad9c1199c476d96f9304d57cecf6c

    {'smithi061.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'CentOS-PowerTools':
    Yum repo downloading error: Downloading error(s): repodata/16bdfd9c9155376d
    dc8f552e5bcb7a6deb94739e35f439ba1a5db715bac5ddd9-primary.xml.gz - Cannot
    download, all mirrors were already tried without success; repodata/25530bb7
    730289658577cd6a7aed7adb251d5fb40a42ebce2cb1f6d1cf1b1f87-filelists.xml.gz -
    Cannot download, all mirrors were already tried without success;
    repodata/38c2c78c3f89e4d347f07263805d33d8ae17df60ebab2e8a259218c57bcae2fb-
    comps-PowerTools.x86_64.xml - Cannot download, all mirrors were already
    tried without success", 'rc': 1, 'results': []}}

[7244289]  rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:04:27
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244289/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244289/
sentry: https://sentry.ceph.com/organizations/ceph/?query=021de0a584e049498ef680584c4fe6a5

    {'smithi090.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    38.145.60.20)", 'rc': 1, 'results': []}}

[7244300]  rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:04:32
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244300/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244300/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d1293ace44d646d4aca6d20d0205788a

    {'smithi103.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    152.19.134.198)", 'rc': 1, 'results': []}}

[7244301]  rados/upgrade/parallel/{0-random-distro$/{rhel_8.6_container_tools_3.0} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}}
-----------------------------------------------------------------
time:   00:04:36
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244301/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244301/
sentry: https://sentry.ceph.com/organizations/ceph/?query=21a6ef75d7f7412ea8cb1828a7d602f7

    {'smithi156.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    67.219.144.68)", 'rc': 1, 'results': []}, 'smithi087.front.sepia.ceph.com':
    {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args':
    {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False,
    'bugfix': False, 'conf_file': None, 'disable_excludes': None,
    'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [],
    'download_dir': None, 'download_only': False, 'enable_plugin': [],
    'enablerepo': [], 'exclude': [], 'install_repoquery': True,
    'install_weak_deps': True, 'installroot': '/', 'list': None,
    'lock_timeout': 30, 'name': ['krb5-workstation'], 'releasever': None,
    'security': False, 'skip_broken': False, 'state': 'present',
    'update_cache': False, 'update_only': False, 'validate_certs': True}},
    'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    152.19.134.142)", 'rc': 1, 'results': []}}

[7244307]  rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0} 1-start 2-services/basic 3-final}
-----------------------------------------------------------------
time:   00:04:40
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244307/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244307/
sentry: https://sentry.ceph.com/organizations/ceph/?query=81d3547571644b05bd545499a548cc3b

    {'smithi060.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    152.19.134.142)", 'rc': 1, 'results': []}}

[7244309]  rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:04:33
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244309/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244309/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d86bcb015f75439c85965940ff1e1e76

    {'smithi155.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    67.219.144.68)", 'rc': 1, 'results': []}}

[7244316]  rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/rados_python}
-----------------------------------------------------------------
time:   00:04:40
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244316/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244316/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1bf86b4c585f4c22b64a1f908bcc3a43

    {'smithi150.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    152.19.134.198)", 'rc': 1, 'results': []},
    'smithi049.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    209.132.190.2)", 'rc': 1, 'results': []}}

[7244317]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/admin_socket_objecter_requests}
-----------------------------------------------------------------
time:   00:04:39
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244317/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244317/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a70d9d74294f40ee94678dd0059b0fa4

    {'smithi090.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    38.145.60.20)", 'rc': 1, 'results': []}, 'smithi131.front.sepia.ceph.com':
    {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args':
    {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False,
    'bugfix': False, 'conf_file': None, 'disable_excludes': None,
    'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [],
    'download_dir': None, 'download_only': False, 'enable_plugin': [],
    'enablerepo': [], 'exclude': [], 'install_repoquery': True,
    'install_weak_deps': True, 'installroot': '/', 'list': None,
    'lock_timeout': 30, 'name': ['krb5-workstation'], 'releasever': None,
    'security': False, 'skip_broken': False, 'state': 'present',
    'update_cache': False, 'update_only': False, 'validate_certs': True}},
    'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    152.19.134.142)", 'rc': 1, 'results': []}}

[7244335]  rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
-----------------------------------------------------------------
time:   00:04:44
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244335/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244335/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c33fd28f2d39476f9febaf505de2ee95

    {'smithi176.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    38.145.60.21)", 'rc': 1, 'results': []}, 'smithi050.front.sepia.ceph.com':
    {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args':
    {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False,
    'bugfix': False, 'conf_file': None, 'disable_excludes': None,
    'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [],
    'download_dir': None, 'download_only': False, 'enable_plugin': [],
    'enablerepo': [], 'exclude': [], 'install_repoquery': True,
    'install_weak_deps': True, 'installroot': '/', 'list': None,
    'lock_timeout': 30, 'name': ['krb5-workstation'], 'releasever': None,
    'security': False, 'skip_broken': False, 'state': 'present',
    'update_cache': False, 'update_only': False, 'validate_certs': True}},
    'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    8.43.85.67)", 'rc': 1, 'results': []}}

[7244336]  rados/singleton/{all/ec-inconsistent-hinfo mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:04:36
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244336/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244336/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4ef245272d724c659810a26c47347b63

    {'smithi179.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    38.145.60.20)", 'rc': 1, 'results': []}}

[7244339]  rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} tasks/mon_recovery}
-----------------------------------------------------------------
time:   00:04:34
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244339/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244339/
sentry: https://sentry.ceph.com/organizations/ceph/?query=df789614f04546aaa2fc6f45db8d4197

    {'smithi188.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    67.219.144.68)", 'rc': 1, 'results': []}, 'smithi138.front.sepia.ceph.com':
    {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args':
    {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False,
    'bugfix': False, 'conf_file': None, 'disable_excludes': None,
    'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [],
    'download_dir': None, 'download_only': False, 'enable_plugin': [],
    'enablerepo': [], 'exclude': [], 'install_repoquery': True,
    'install_weak_deps': True, 'installroot': '/', 'list': None,
    'lock_timeout': 30, 'name': ['krb5-workstation'], 'releasever': None,
    'security': False, 'skip_broken': False, 'state': 'present',
    'update_cache': False, 'update_only': False, 'validate_certs': True}},
    'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    38.145.60.21)", 'rc': 1, 'results': []}}

[7244344]  rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_nfs}
-----------------------------------------------------------------
time:   00:04:34
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244344/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244344/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0f0ee3467fa74bf5a3f36e0403eb2aa3

    {'smithi176.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    152.19.134.198)", 'rc': 1, 'results': []}}

[7244345]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache}
-----------------------------------------------------------------
time:   00:04:38
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244345/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244345/
sentry: https://sentry.ceph.com/organizations/ceph/?query=cd7ac55319bb443f927c4393cf700b43

    {'smithi179.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    8.43.85.67)", 'rc': 1, 'results': []}, 'smithi050.front.sepia.ceph.com':
    {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args':
    {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False,
    'bugfix': False, 'conf_file': None, 'disable_excludes': None,
    'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [],
    'download_dir': None, 'download_only': False, 'enable_plugin': [],
    'enablerepo': [], 'exclude': [], 'install_repoquery': True,
    'install_weak_deps': True, 'installroot': '/', 'list': None,
    'lock_timeout': 30, 'name': ['krb5-workstation'], 'releasever': None,
    'security': False, 'skip_broken': False, 'state': 'present',
    'update_cache': False, 'update_only': False, 'validate_certs': True}},
    'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    8.43.85.67)", 'rc': 1, 'results': []}}

[7244355]  rados/singleton-nomsgr/{all/crushdiff mon_election/classic rados supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:04:37
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244355/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244355/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f55cbcbc482244708e7c6235ba005b3e

    {'smithi167.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    140.211.169.196)", 'rc': 1, 'results': []}}

[7244359]  rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start}
-----------------------------------------------------------------
time:   00:04:38
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244359/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244359/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9d3cdd05c0a04db0b38d542dcce6d6f6

    {'smithi087.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    38.145.60.21)", 'rc': 1, 'results': []}, 'smithi184.front.sepia.ceph.com':
    {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args':
    {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False,
    'bugfix': False, 'conf_file': None, 'disable_excludes': None,
    'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [],
    'download_dir': None, 'download_only': False, 'enable_plugin': [],
    'enablerepo': [], 'exclude': [], 'install_repoquery': True,
    'install_weak_deps': True, 'installroot': '/', 'list': None,
    'lock_timeout': 30, 'name': ['krb5-workstation'], 'releasever': None,
    'security': False, 'skip_broken': False, 'state': 'present',
    'update_cache': False, 'update_only': False, 'validate_certs': True}},
    'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    152.19.134.142)", 'rc': 1, 'results': []}}

[7244365]  rados/standalone/{supported-random-distro$/{rhel_8} workloads/osd-backfill}
-----------------------------------------------------------------
time:   00:04:32
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244365/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244365/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5ae1ff82f26d446187c156d2d9104a89

    {'smithi195.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    38.145.60.20)", 'rc': 1, 'results': []}}

[7244371]  rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
-----------------------------------------------------------------
time:   00:04:47
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244371/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244371/
sentry: https://sentry.ceph.com/organizations/ceph/?query=44155dc5ea5741c1a89799bb00ec444f

    {'smithi167.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    38.145.60.20)", 'rc': 1, 'results': []}, 'smithi158.front.sepia.ceph.com':
    {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args':
    {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False,
    'bugfix': False, 'conf_file': None, 'disable_excludes': None,
    'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [],
    'download_dir': None, 'download_only': False, 'enable_plugin': [],
    'enablerepo': [], 'exclude': [], 'install_repoquery': True,
    'install_weak_deps': True, 'installroot': '/', 'list': None,
    'lock_timeout': 30, 'name': ['krb5-workstation'], 'releasever': None,
    'security': False, 'skip_broken': False, 'state': 'present',
    'update_cache': False, 'update_only': False, 'validate_certs': True}},
    'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    152.19.134.142)", 'rc': 1, 'results': []},
    'smithi019.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    67.219.144.68)", 'rc': 1, 'results': []}, 'smithi012.front.sepia.ceph.com':
    {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args':
    {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False,
    'bugfix': False, 'conf_file': None, 'disable_excludes': None,
    'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [],
    'download_dir': None, 'download_only': False, 'enable_plugin': [],
    'enablerepo': [], 'exclude': [], 'install_repoquery': True,
    'install_weak_deps': True, 'installroot': '/', 'list': None,
    'lock_timeout': 30, 'name': ['krb5-workstation'], 'releasever': None,
    'security': False, 'skip_broken': False, 'state': 'present',
    'update_cache': False, 'update_only': False, 'validate_certs': True}},
    'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    67.219.144.68)", 'rc': 1, 'results': []}}

[7244386]  rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add}
-----------------------------------------------------------------
time:   00:04:40
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244386/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244386/
sentry: https://sentry.ceph.com/organizations/ceph/?query=14fa8e4f6b2546398fb51a6bb24d91b1

    {'smithi155.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    34.221.3.152)", 'rc': 1, 'results': []}, 'smithi017.front.sepia.ceph.com':
    {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args':
    {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False,
    'bugfix': False, 'conf_file': None, 'disable_excludes': None,
    'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [],
    'download_dir': None, 'download_only': False, 'enable_plugin': [],
    'enablerepo': [], 'exclude': [], 'install_repoquery': True,
    'install_weak_deps': True, 'installroot': '/', 'list': None,
    'lock_timeout': 30, 'name': ['krb5-workstation'], 'releasever': None,
    'security': False, 'skip_broken': False, 'state': 'present',
    'update_cache': False, 'update_only': False, 'validate_certs': True}},
    'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    34.221.3.152)", 'rc': 1, 'results': []}}

[7244387]  rados/objectstore/{backends/objectstore-bluestore-b supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:04:39
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244387/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244387/
sentry: https://sentry.ceph.com/organizations/ceph/?query=497c6491198142afbdda5092e94b9657

    {'smithi049.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    8.43.85.73)", 'rc': 1, 'results': []}}

[7244388]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/small-objects}
-----------------------------------------------------------------
time:   00:04:35
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244388/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244388/
sentry: https://sentry.ceph.com/organizations/ceph/?query=26f4568392854b259cd8006620773be1

    {'smithi122.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    152.19.134.198)", 'rc': 1, 'results': []},
    'smithi116.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    152.19.134.142)", 'rc': 1, 'results': []}}

[7244393]  rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_orch_cli_mon}
-----------------------------------------------------------------
time:   00:38:56
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244393/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244393/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e3b323c9cb474fde8dd389b5ceafda50

    {'smithi184.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'CentOS-PowerTools':
    Yum repo downloading error: Downloading error(s): repodata/16bdfd9c9155376d
    dc8f552e5bcb7a6deb94739e35f439ba1a5db715bac5ddd9-primary.xml.gz - Cannot
    download, all mirrors were already tried without success; repodata/25530bb7
    730289658577cd6a7aed7adb251d5fb40a42ebce2cb1f6d1cf1b1f87-filelists.xml.gz -
    Cannot download, all mirrors were already tried without success;
    repodata/38c2c78c3f89e4d347f07263805d33d8ae17df60ebab2e8a259218c57bcae2fb-
    comps-PowerTools.x86_64.xml - Cannot download, all mirrors were already
    tried without success", 'rc': 1, 'results': []}}

[7244400]  rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:04:29
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244400/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244400/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8fbfeb34e52740419746f9cc9c5af3c5

    {'smithi173.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    209.132.190.2)", 'rc': 1, 'results': []}}

[7244404]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/admin_socket_objecter_requests}
-----------------------------------------------------------------
time:   00:04:38
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244404/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244404/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e460d967ec5f41b98351bed241d5f8bd

    {'smithi172.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    209.132.190.2)", 'rc': 1, 'results': []}, 'smithi019.front.sepia.ceph.com':
    {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args':
    {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False,
    'bugfix': False, 'conf_file': None, 'disable_excludes': None,
    'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [],
    'download_dir': None, 'download_only': False, 'enable_plugin': [],
    'enablerepo': [], 'exclude': [], 'install_repoquery': True,
    'install_weak_deps': True, 'installroot': '/', 'list': None,
    'lock_timeout': 30, 'name': ['krb5-workstation'], 'releasever': None,
    'security': False, 'skip_broken': False, 'state': 'present',
    'update_cache': False, 'update_only': False, 'validate_certs': True}},
    'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    152.19.134.198)", 'rc': 1, 'results': []}}

[7244405]  rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-many-deletes}
-----------------------------------------------------------------
time:   00:04:43
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244405/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244405/
sentry: https://sentry.ceph.com/organizations/ceph/?query=16445a6719704b969eb1deefef59381e

    {'smithi131.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    140.211.169.196)", 'rc': 1, 'results': []},
    'smithi032.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    140.211.169.196)", 'rc': 1, 'results': []}}

[7244410]  rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:04:36
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244410/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244410/
sentry: https://sentry.ceph.com/organizations/ceph/?query=704b8ace6b8a4f5d97953a909f7862fb

    {'smithi032.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    8.43.85.73)", 'rc': 1, 'results': []}}

[7244414]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/cache-agent-small}
-----------------------------------------------------------------
time:   00:04:33
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244414/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244414/
sentry: https://sentry.ceph.com/organizations/ceph/?query=3d5fd3481e7c4cb49063469b647e47ff

    {'smithi167.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    152.19.134.142)", 'rc': 1, 'results': []},
    'smithi174.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    209.132.190.2)", 'rc': 1, 'results': []}}

[7244420]  rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:04:42
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244420/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244420/
sentry: https://sentry.ceph.com/organizations/ceph/?query=29d4e0b1a0e64916b554b67a1bc8bdf7

    {'smithi012.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    67.219.144.68)", 'rc': 1, 'results': []}}

[7244426]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-snaps}
-----------------------------------------------------------------
time:   00:04:37
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244426/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244426/
sentry: https://sentry.ceph.com/organizations/ceph/?query=3010798445c14804b3df0bffce1fdaef

    {'smithi120.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    67.219.144.68)", 'rc': 1, 'results': []}, 'smithi002.front.sepia.ceph.com':
    {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args':
    {'allow_downgrade': False, 'allowerasing': False, 'autoremove': False,
    'bugfix': False, 'conf_file': None, 'disable_excludes': None,
    'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [],
    'download_dir': None, 'download_only': False, 'enable_plugin': [],
    'enablerepo': [], 'exclude': [], 'install_repoquery': True,
    'install_weak_deps': True, 'installroot': '/', 'list': None,
    'lock_timeout': 30, 'name': ['krb5-workstation'], 'releasever': None,
    'security': False, 'skip_broken': False, 'state': 'present',
    'update_cache': False, 'update_only': False, 'validate_certs': True}},
    'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    8.43.85.73)", 'rc': 1, 'results': []}}

[7244427]  rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-flag}
-----------------------------------------------------------------
time:   00:04:36
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244427/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244427/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ce29c21d683047aa9dcfaec8530b9ad4

    {'smithi174.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    140.211.169.196)", 'rc': 1, 'results': []},
    'smithi167.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    34.221.3.152)", 'rc': 1, 'results': []}}

[7244435]  rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_cephadm}
-----------------------------------------------------------------
time:   00:04:30
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244435/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244435/
sentry: https://sentry.ceph.com/organizations/ceph/?query=35632fc64235456181b8f028e35aed87

    {'smithi187.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    140.211.169.196)", 'rc': 1, 'results': []}}

[7244439]  rados/singleton/{all/mon-config mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:04:27
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244439/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244439/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e2630773e26849d1b805a8bbcb9c351a

    {'smithi202.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    38.145.60.21)", 'rc': 1, 'results': []}}

[7244441]  rados/objectstore/{backends/objectstore-memstore supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:04:26
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244441/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244441/
sentry: https://sentry.ceph.com/organizations/ceph/?query=048d549fcb2247b5adffb456c8802980

    {'smithi141.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed':
    False, 'invocation': {'module_args': {'allow_downgrade': False,
    'allowerasing': False, 'autoremove': False, 'bugfix': False, 'conf_file':
    None, 'disable_excludes': None, 'disable_gpg_check': False,
    'disable_plugin': [], 'disablerepo': [], 'download_dir': None,
    'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude':
    [], 'install_repoquery': True, 'install_weak_deps': True, 'installroot':
    '/', 'list': None, 'lock_timeout': 30, 'name': ['krb5-workstation'],
    'releasever': None, 'security': False, 'skip_broken': False, 'state':
    'present', 'update_cache': False, 'update_only': False, 'validate_certs':
    True}}, 'msg': "Failed to download metadata for repo 'epel': Cannot prepare
    internal mirrorlist: Status code: 404 for https://mirrors.fedoraproject.org
    /metalink?repo=epel-8.6&arch=x86_64&infra=$infra&content=$contentdir (IP:
    38.145.60.21)", 'rc': 1, 'results': []}}



Pass
=================================================================
[7244059] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/write_fadvise_dontneed}
time:   00:15:05
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244059/

[7244062] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/scrub}
time:   02:47:45
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244062/

[7244065] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/admin_socket_objecter_requests}
time:   00:24:12
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244065/

[7244066] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-small-objects-many-deletes}
time:   00:22:07
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244066/

[7244067] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-agent-big}
time:   00:37:10
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244067/

[7244068] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:25:10
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244068/

[7244070] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read}
time:   00:10:49
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244070/

[7244073] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add}
time:   00:17:58
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244073/

[7244083] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/cache-snaps-balanced}
time:   00:23:47
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244083/

[7244086] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache}
time:   00:13:38
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244086/

[7244087] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start}
time:   00:23:44
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244087/

[7244088] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04}}
time:   00:22:02
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244088/

[7244091] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{ubuntu_20.04}}
time:   00:12:10
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244091/

[7244093] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read}
time:   00:10:52
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244093/

[7244096] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} thrashers/none thrashosds-health workloads/pool-snaps-few-objects}
time:   00:25:46
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244096/

[7244100] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/radosbench-high-concurrency}
time:   00:40:23
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244100/

[7244107] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{ubuntu_20.04}}
time:   00:18:27
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244107/

[7244108] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/c2c}
time:   00:13:45
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244108/

[7244109] rados/upgrade/parallel/{0-random-distro$/{ubuntu_20.04} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}}
time:   01:35:41
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244109/

[7244111] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/fastread thrashosds-health workloads/ec-small-objects}
time:   00:20:50
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244111/

[7244112] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/radosbench}
time:   00:55:38
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244112/

[7244113] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mostlyread}
time:   00:34:06
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244113/

[7244114] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04}}
time:   00:15:08
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244114/

[7244120] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2}
time:   00:25:42
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244120/

[7244128] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read}
time:   00:10:49
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244128/

[7244129] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/small-objects-balanced}
time:   00:22:03
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244129/

[7244131] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/small-objects-localized}
time:   00:24:07
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244131/

[7244133] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_cephadm_repos}
time:   00:06:49
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244133/

[7244137] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}}
time:   00:09:29
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244137/

[7244138] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/snaps-few-objects-localized}
time:   00:26:49
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244138/

[7244141] rados/singleton/{all/radostool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04}}
time:   00:15:01
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244141/

[7244143] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/write_fadvise_dontneed}
time:   00:16:20
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244143/

[7244146] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}}
time:   00:23:02
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244146/

[7244147] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04}}
time:   00:32:29
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244147/

[7244148] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{ubuntu_20.04}}
time:   00:08:14
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244148/

[7244150] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/admin_socket_objecter_requests}
time:   00:22:55
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244150/

[7244151] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2}
time:   00:25:08
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244151/

[7244153] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read}
time:   00:10:44
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244153/

[7244155] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:25:26
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244155/

[7244156] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:12:06
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244156/

[7244157] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites}
time:   00:21:55
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244157/

[7244160] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}
time:   00:13:08
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244160/

[7244166] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} tasks/mon_clock_with_skews}
time:   00:08:26
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244166/

[7244171] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/repair_test}
time:   00:18:54
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244171/

[7244173] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{ubuntu_latest}}
time:   00:10:26
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244173/

[7244174] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache}
time:   00:14:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244174/

[7244177] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write}
time:   00:10:42
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244177/

[7244178] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/pool-create-delete}
time:   00:17:06
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244178/

[7244182] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}
time:   00:10:47
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244182/

[7244183] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/pool-snaps-few-objects}
time:   00:34:34
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244183/

[7244188] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/radosbench-high-concurrency}
time:   00:34:07
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244188/

[7244190] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{ubuntu_20.04}}
time:   00:10:10
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244190/

[7244191] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/erasure-code}
time:   01:25:51
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244191/

[7244192] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/radosbench}
time:   00:27:47
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244192/

[7244194] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:13:17
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244194/

[7244195] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/redirect}
time:   00:12:31
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244195/

[7244206] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write}
time:   00:25:28
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244206/

[7244208] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/set-chunks-read}
time:   00:19:50
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244208/

[7244209] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery}
time:   00:17:18
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244209/

[7244212] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites}
time:   00:22:56
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244212/

[7244213] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/small-objects-balanced}
time:   00:21:51
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244213/

[7244214] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/small-objects-localized}
time:   00:22:15
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244214/

[7244215] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{ubuntu_20.04}}
time:   00:10:37
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244215/

[7244217] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/small-objects}
time:   00:22:25
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244217/

[7244221] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} thrashers/force-sync-many workloads/rados_5925}
time:   00:11:20
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244221/

[7244226] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/snaps-few-objects}
time:   00:27:29
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244226/

[7244227] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite}
time:   00:16:48
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244227/

[7244229] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{ubuntu_latest} tasks/module_selftest}
time:   00:27:46
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244229/

[7244230] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio}
time:   00:12:41
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244230/

[7244232] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/admin_socket_objecter_requests}
time:   00:16:56
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244232/

[7244233] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1}
time:   00:28:31
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244233/

[7244234] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{ubuntu_20.04}}
time:   00:19:02
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244234/

[7244235] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/mgr}
time:   00:17:24
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244235/

[7244238] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/cache-agent-big}
time:   00:21:39
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244238/

[7244240] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:13:59
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244240/

[7244241] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:24:46
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244241/

[7244242] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} thrashers/none thrashosds-health workloads/cache-agent-small}
time:   00:12:37
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244242/

[7244243] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04}}
time:   00:33:21
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244243/

[7244244] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps-readproxy}
time:   00:23:17
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244244/

[7244246] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04}}
time:   00:21:02
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244246/

[7244247] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}}
time:   00:21:49
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244247/

[7244248] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-pool-snaps}
time:   00:28:47
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244248/

[7244249] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_adoption}
time:   00:09:59
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244249/

[7244255] rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04}}
time:   00:09:13
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244255/

[7244256] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag}
time:   00:18:07
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244256/

[7244257] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{ubuntu_latest}}
time:   00:20:44
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244257/

[7244258] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/cache}
time:   00:12:05
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244258/

[7244259] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} tasks/rados_api_tests}
time:   01:26:28
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244259/

[7244260] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench}
time:   00:10:47
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244260/

[7244261] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/dedup-io-mixed}
time:   00:14:19
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244261/

[7244262] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/many workloads/rados_api_tests}
time:   02:00:08
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244262/

[7244263] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/dedup-io-snaps}
time:   00:12:52
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244263/

[7244264] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites}
time:   00:25:02
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244264/

[7244266] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:50:10
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244266/

[7244269] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rados_api_tests}
time:   00:31:29
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244269/

[7244270] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_latest} tasks/progress}
time:   00:25:18
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244270/

[7244271] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{ubuntu_latest}}
time:   00:39:21
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244271/

[7244272] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_cephadm}
time:   01:39:25
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244272/

[7244274] rados/singleton/{all/admin-socket mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04}}
time:   00:08:46
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244274/

[7244275] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-radosbench}
time:   01:44:39
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244275/

[7244276] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/radosbench}
time:   00:57:38
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244276/

[7244277] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{centos_8}}
time:   01:49:47
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244277/

[7244282] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait}
time:   00:54:57
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244282/

[7244284] rados/singleton/{all/backfill-toofull mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}}
time:   01:42:26
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244284/

[7244285] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read}
time:   00:12:37
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244285/

[7244287] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} tasks/rados_cls_all}
time:   01:16:53
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244287/

[7244288] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/redirect_set_object}
time:   00:16:01
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244288/

[7244290] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:50:27
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244290/

[7244291] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/set-chunks-read}
time:   00:14:39
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244291/

[7244292] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews}
time:   01:34:59
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244292/

[7244293] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind}
time:   02:09:25
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244293/

[7244294] rados/singleton/{all/deduptool mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}}
time:   00:12:20
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244294/

[7244295] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/small-objects-balanced}
time:   01:45:21
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244295/

[7244296] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/small-objects-localized}
time:   00:20:44
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244296/

[7244297] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_cephadm_repos}
time:   00:43:58
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244297/

[7244298] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-bitmap} tasks/dashboard}
time:   02:31:43
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244298/

[7244302] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} thrashers/none thrashosds-health workloads/small-objects}
time:   00:20:04
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244302/

[7244303] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/snaps-few-objects}
time:   02:03:30
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244303/

[7244304] rados/singleton/{all/divergent_priors mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04}}
time:   00:10:08
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244304/

[7244305] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/snaps-few-objects-balanced}
time:   01:52:40
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244305/

[7244306] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/one workloads/rados_mon_osdmap_prune}
time:   02:26:06
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244306/

[7244308] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/snaps-few-objects-localized}
time:   00:29:10
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244308/

[7244310] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/snaps-few-objects}
time:   01:50:40
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244310/

[7244311] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw}
time:   00:12:25
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244311/

[7244312] rados/singleton/{all/divergent_priors2 mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04}}
time:   00:10:16
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244312/

[7244313] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{ubuntu_latest} tasks/prometheus}
time:   00:13:50
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244313/

[7244314] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/write_fadvise_dontneed}
time:   00:17:06
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244314/

[7244318] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-small-objects-balanced}
time:   01:46:46
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244318/

[7244320] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites}
time:   00:42:20
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244320/

[7244322] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} thrashers/none thrashosds-health workloads/cache-agent-big}
time:   00:16:46
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244322/

[7244324] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}}
time:   00:11:40
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244324/

[7244325] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/mon}
time:   01:18:27
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244325/

[7244326] rados/singleton/{all/dump-stuck mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}}
time:   01:32:31
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244326/

[7244327] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:29:15
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244327/

[7244328] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:14:17
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244328/

[7244329] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rmdir-reactivate}
time:   00:49:08
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244329/

[7244330] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-agent-small}
time:   00:15:09
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244330/

[7244331] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache-pool-snaps-readproxy}
time:   01:56:29
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244331/

[7244332] rados/objectstore/{backends/objectstore-bluestore-a supported-random-distro$/{ubuntu_latest}}
time:   03:12:03
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244332/

[7244333] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}
time:   00:13:51
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244333/

[7244334] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/cache-pool-snaps}
time:   00:23:23
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244334/

[7244337] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{centos_8}}
time:   01:29:30
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244337/

[7244338] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache-snaps-balanced}
time:   01:49:42
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244338/

[7244340] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados tasks/rados_cls_all validater/lockdep}
time:   01:44:27
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244340/

[7244341] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04}}
time:   00:33:07
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244341/

[7244342] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/cache-snaps}
time:   00:22:54
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244342/

[7244343] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read}
time:   00:12:30
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244343/

[7244346] rados/singleton/{all/ec-lost-unfound mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04}}
time:   00:36:23
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244346/

[7244347] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}
time:   00:09:54
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244347/

[7244348] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/dedup-io-mixed}
time:   00:13:20
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244348/

[7244349] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} tasks/rados_stress_watch}
time:   00:19:16
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244349/

[7244350] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/sync-many workloads/rados_mon_workunits}
time:   00:31:36
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244350/

[7244351] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/dedup-io-snaps}
time:   01:36:36
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244351/

[7244352] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/repave-all}
time:   00:52:30
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244352/

[7244353] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/pool-snaps-few-objects}
time:   00:32:33
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244353/

[7244354] rados/singleton/{all/erasure-code-nonregression mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}}
time:   01:30:20
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244354/

[7244356] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/test_rbd_api}
time:   01:42:03
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244356/

[7244357] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/rados_api_tests}
time:   00:39:40
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244357/

[7244358] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{ubuntu_20.04} tasks/workunits}
time:   00:09:52
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244358/

[7244360] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/radosbench-high-concurrency}
time:   02:02:57
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244360/

[7244361] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-fast-read}
time:   00:22:29
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244361/

[7244362] rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}}
time:   01:49:50
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244362/

[7244363] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} thrashers/none thrashosds-health workloads/radosbench}
time:   00:44:40
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244363/

[7244364] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{centos_8}}
time:   01:35:25
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244364/

[7244366] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/redirect}
time:   00:13:00
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244366/

[7244367] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest}
time:   01:45:13
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244367/

[7244368] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw}
time:   00:12:27
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244368/

[7244369] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_orch_cli}
time:   00:19:20
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244369/

[7244370] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   01:43:55
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244370/

[7244372] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/redirect_promote_tests}
time:   00:20:32
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244372/

[7244373] rados/singleton/{all/lost-unfound mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}}
time:   02:00:57
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244373/

[7244374] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/redirect_set_object}
time:   01:41:02
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244374/

[7244375] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2}
time:   00:27:10
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244375/

[7244376] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites}
time:   00:21:07
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244376/

[7244377] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:25:24
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244377/

[7244378] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/set-chunks-read}
time:   00:18:27
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244378/

[7244379] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/rados_striper}
time:   01:06:18
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244379/

[7244380] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{ubuntu_latest}}
time:   00:11:06
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244380/

[7244381] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/small-objects-balanced}
time:   00:21:03
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244381/

[7244382] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews}
time:   01:35:06
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244382/

[7244383] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados tasks/mon_recovery validater/lockdep}
time:   01:37:55
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244383/

[7244384] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}}
time:   00:10:38
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244384/

[7244385] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/small-objects-localized}
time:   00:21:54
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244385/

[7244389] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}
time:   00:12:06
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244389/

[7244390] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/snaps-few-objects-balanced}
time:   01:50:34
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244390/

[7244391] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}}
time:   01:35:28
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244391/

[7244392] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} thrashers/sync workloads/snaps-few-objects}
time:   00:27:59
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244392/

[7244394] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/snaps-few-objects-localized}
time:   00:27:17
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244394/

[7244395] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write}
time:   00:12:22
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244395/

[7244396] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects}
time:   01:56:32
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244396/

[7244398] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-bitmap} supported-random-distro$/{ubuntu_20.04} tasks/crash}
time:   00:11:38
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244398/

[7244401] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/write_fadvise_dontneed}
time:   00:17:17
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244401/

[7244402] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start}
time:   00:51:06
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244402/

[7244403] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}}
time:   00:14:26
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244403/

[7244406] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_big}
time:   01:26:52
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244406/

[7244407] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/cache-agent-big}
time:   00:19:26
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244407/

[7244408] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/cache-snaps}
time:   01:44:46
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244408/

[7244409] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools} 1-start 2-services/rgw 3-final}
time:   00:54:55
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244409/

[7244411] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd}
time:   02:59:21
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244411/

[7244412] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:31:52
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244412/

[7244413] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:12:08
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244413/

[7244415] rados/singleton/{all/mon-auth-caps mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04}}
time:   00:11:06
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244415/

[7244416] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy}
time:   00:24:13
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244416/

[7244417] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}
time:   00:11:32
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244417/

[7244418] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps}
time:   01:43:25
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244418/

[7244419] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_adoption}
time:   01:37:04
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244419/

[7244421] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-snaps-balanced}
time:   01:53:36
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244421/

[7244422] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read}
time:   00:10:51
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244422/

[7244423] rados/singleton/{all/mon-config-key-caps mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04}}
time:   00:10:55
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244423/

[7244424] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} tasks/mon_clock_with_skews}
time:   00:08:38
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244424/

[7244425] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados tasks/rados_api_tests validater/valgrind}
time:   02:19:02
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244425/

[7244428] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache}
time:   00:14:42
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244428/

[7244429] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-small-objects-overwrites}
time:   00:21:23
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244429/

[7244430] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/dedup-io-mixed}
time:   01:33:42
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244430/

[7244431] rados/singleton/{all/mon-config-keys mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}}
time:   00:16:02
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244431/

[7244432] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04}}
time:   00:33:30
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244432/

[7244433] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}}
time:   00:18:46
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244433/

[7244434] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/dedup-io-snaps}
time:   00:13:48
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244434/

[7244436] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/pool-create-delete}
time:   01:37:26
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244436/

[7244437] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mix}
time:   00:23:08
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244437/

[7244438] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/pool-snaps-few-objects}
time:   01:52:08
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244438/

[7244440] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{ubuntu_latest} tasks/failover}
time:   00:14:48
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244440/

[7244442] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/rados_api_tests}
time:   01:53:25
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244442/

[7244444] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start}
time:   00:49:55
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244444/

[7244445] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/radosbench-high-concurrency}
time:   02:08:15
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244445/

[7244446] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects}
time:   00:23:33
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244446/

[7244447] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/radosbench}
time:   00:47:32
info:   http://pulpito.ceph.com/yuriw-2023-04-17_14:26:55-rados-reef-distro-default-smithi/7244447/