2023-11-04T08:57:29.469 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2
2023-11-04T08:57:29.488 INFO:teuthology.results:subset: '111/120000'
2023-11-04T08:57:29.488 INFO:teuthology.results:seed: '6445'
2023-11-04T16:45:25.346 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2
2023-11-04T16:45:25.382 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-11-05T03:47:27.239 INFO:teuthology.results:Tests finished! gathering results...
2023-11-05T03:47:50.785 INFO:scrape:Found 309 jobs
2023-11-05T03:47:50.786 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446583/teuthology.log
2023-11-05T03:47:51.327 WARNING:scrape:Misunderstood line: 
2023-11-05T03:47:53.446 WARNING:scrape:Misunderstood line: 
2023-11-05T03:47:54.104 WARNING:scrape:Misunderstood line: 
2023-11-05T03:47:54.426 WARNING:scrape:Misunderstood line: 
2023-11-05T03:47:55.769 INFO:scrape:Found 13 distinct failure reasons
2023-11-05T03:47:55.769 INFO:scrape:Dead: None
2023-11-05T03:47:55.770 INFO:scrape:13 jobs: ['7446583', '7446394', '7446312', '7446354', '7446348', '7446296', '7446328', '7446342', '7446306', '7446370', '7446410', '7446380', '7446358']
2023-11-05T03:47:55.770 INFO:scrape:
2023-11-05T03:47:55.771 INFO:scrape:Failure: Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_lazy_omap_stats'
2023-11-05T03:47:55.771 INFO:scrape:2 jobs: ['7446140', '7446445']
2023-11-05T03:47:55.772 INFO:scrape:suites intersection: ['rados', 'rados/singleton-nomsgr/{all/lazy_omap_stats_output', 'supported-random-distro$/{ubuntu_latest}}']
2023-11-05T03:47:55.772 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados', 'rados/singleton-nomsgr/{all/lazy_omap_stats_output', 'supported-random-distro$/{ubuntu_latest}}']
2023-11-05T03:47:55.773 INFO:scrape:
2023-11-05T03:47:55.773 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/cls/test_cls_lock.sh
2023-11-05T03:47:55.774 INFO:scrape:1 jobs: ['7446578']
2023-11-05T03:47:55.774 INFO:scrape:suites: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async-v1only', 'objectstore/bluestore-comp-zstd', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_cls_all', 'validater/valgrind}']
2023-11-05T03:47:55.775 INFO:scrape:
2023-11-05T03:47:55.775 INFO:scrape:Valgrind: mon (Leak_StillReachable)
2023-11-05T03:47:55.775 INFO:scrape:4 jobs: ['7446467', '7446469', '7446200', '7446346']
2023-11-05T03:47:55.776 INFO:scrape:suites intersection: []
2023-11-05T03:47:55.776 INFO:scrape:suites union: ['2-inject-leak/none', 'centos_latest}', 'ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/valgrind-leaks/{1-start', 'rados/verify/{centos_latest', 'tasks/mon_recovery', 'tasks/rados_api_tests', 'tasks/rados_cls_all', 'validater/valgrind}']
2023-11-05T03:47:55.777 INFO:scrape:
2023-11-05T03:47:55.777 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi057 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=23776148d05fd5bbb0eae6dde0ebb81fe2ceda46 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh'
2023-11-05T03:47:55.778 INFO:scrape:2 jobs: ['7446417', '7446660']
2023-11-05T03:47:55.778 INFO:scrape:suites intersection: ['debug/mgr', 'rados/dashboard/{0-single-container-host', 'tasks/e2e}']
2023-11-05T03:47:55.779 INFO:scrape:suites union: ['debug/mgr', 'mon_election/classic', 'mon_election/connectivity', 'rados/dashboard/{0-single-container-host', 'random-objectstore$/{bluestore-comp-zlib}', 'random-objectstore$/{bluestore-low-osd-mem-target}', 'tasks/e2e}']
2023-11-05T03:47:55.779 INFO:scrape:
2023-11-05T03:47:55.779 INFO:scrape:Failure: Command failed (workunit test scrub/osd-recovery-scrub.sh) on smithi152 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=23776148d05fd5bbb0eae6dde0ebb81fe2ceda46 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh'
2023-11-05T03:47:55.780 INFO:scrape:1 jobs: ['7446210']
2023-11-05T03:47:55.780 INFO:scrape:suites: ['rados/standalone/{supported-random-distro$/{ubuntu_latest}', 'workloads/scrub}']
2023-11-05T03:47:55.781 INFO:scrape:
2023-11-05T03:47:55.781 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/cls/test_cls_2pc_queue.sh
2023-11-05T03:47:55.782 INFO:scrape:1 jobs: ['7446278']
2023-11-05T03:47:55.782 INFO:scrape:suites: ['0-start', '1-tasks', 'mon_election/classic', 'rados/upgrade/parallel/{0-random-distro$/{rhel_8.6_container_tools_3.0}', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'test_rbd_api', 'test_rbd_python}}', 'upgrade-sequence', 'workload/{ec-rados-default']
2023-11-05T03:47:55.783 INFO:scrape:
2023-11-05T03:47:55.783 INFO:scrape:Failure: Command failed on smithi159 with status 100: 'sudo apt-get clean'
2023-11-05T03:47:55.784 INFO:scrape:1 jobs: ['7446298']
2023-11-05T03:47:55.784 INFO:scrape:suites: ['clusters/3-mons', 'mon_election/classic', 'msgr-failures/mon-delay', 'msgr/async', 'objectstore/bluestore-comp-snappy', 'rados', 'rados/monthrash/{ceph', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/one', 'workloads/snaps-few-objects}']
2023-11-05T03:47:55.784 INFO:scrape:
2023-11-05T03:47:55.785 INFO:scrape:Failure: Command failed on smithi164 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff5e4048-7b21-11ee-8db9-212e2dc638e7 -- ceph mon dump -f json'
2023-11-05T03:47:55.785 INFO:scrape:1 jobs: ['7446562']
2023-11-05T03:47:55.786 INFO:scrape:suites: ['0-start', '1-tasks', 'mon_election/connectivity', 'rados/upgrade/parallel/{0-random-distro$/{rhel_8.6_container_tools_3.0}', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'test_rbd_api', 'test_rbd_python}}', 'upgrade-sequence', 'workload/{ec-rados-default']
2023-11-05T03:47:55.786 INFO:scrape:
2023-11-05T03:47:55.787 INFO:scrape:Dead: b'2023-11-05T02:37:46.231 DEBUG:teuthology.exit:Finished running handlers'
2023-11-05T03:47:55.787 INFO:scrape:3 jobs: ['7446558', '7446582', '7446274']
2023-11-05T03:47:55.787 INFO:scrape:suites intersection: []
2023-11-05T03:47:55.788 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados', 'rados/objectstore/{backends/objectstore-bluestore-a', 'rados/singleton-nomsgr/{all/admin_socket_output', 'supported-random-distro$/{centos_latest}}', 'supported-random-distro$/{ubuntu_latest}}']
2023-11-05T03:47:55.788 INFO:scrape:
2023-11-05T03:47:55.789 INFO:scrape:Failure: SELinux denials found on ubuntu@smithi031.front.sepia.ceph.com: ['type=AVC msg=audit(1699094072.281:21965): avc:  denied  { ioctl } for  pid=125244 comm="iptables" path="/var/lib/containers/storage/overlay/c324b2e1cdb2db590ae5e9d0b4bcf87178fe5194488b37099d17e7508c4a5683/merged" dev="overlay" ino=2228302 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1']
2023-11-05T03:47:55.789 INFO:scrape:1 jobs: ['7446207']
2023-11-05T03:47:55.789 INFO:scrape:suites: ['agent/off', 'mon_election/connectivity', 'rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8', 'task/test_extra_daemon_features}']
2023-11-05T03:47:55.790 INFO:scrape:
2023-11-05T03:47:55.790 INFO:scrape:Failure: No module named 'tasks.cephadm'
2023-11-05T03:47:55.791 INFO:scrape:3 jobs: ['7446308', '7446338', '7446314']
2023-11-05T03:47:55.791 INFO:scrape:suites intersection: ['mon_election/classic']
2023-11-05T03:47:55.792 INFO:scrape:suites union: ['agent/on', 'ceph', 'clusters/{fixed-2', 'fast/fast', 'mon_election/classic', 'msgr-failures/osd-delay', 'openstack}', 'rados', 'rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun', 'rados/singleton-nomsgr/{all/ceph-post-file', 'rados/thrash-erasure-code-overwrites/{bluestore-bitmap', 'recovery-overrides/{more-async-recovery}', 'supported-random-distro$/{ubuntu_latest}', 'supported-random-distro$/{ubuntu_latest}}', 'task/test_orch_cli_mon}', 'thrashers/morepggrow', 'thrashosds-health', 'workloads/ec-small-objects-fast-read-overwrites}']
2023-11-05T03:47:55.792 INFO:scrape:
2023-11-05T03:47:55.793 INFO:scrape:Failure: Config file not found: "/home/teuthworker/src/github.com_ceph_ceph-c_23776148d05fd5bbb0eae6dde0ebb81fe2ceda46/qa/tasks/cephadm.conf".
2023-11-05T03:47:55.793 INFO:scrape:1 jobs: ['7446318']
2023-11-05T03:47:55.793 INFO:scrape:suites: ['0-nvme-loop', '1-start', '2-ops/rm-zap-wait}', 'rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun']
2023-11-05T03:47:55.794 INFO:scrape:
2023-11-05T03:47:55.804 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/
failed:  18
dead:    16
running: 0
waiting: 0
queued:  0
passed:  275


Fail
=================================================================
[7446140]  rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:11:13
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446140/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446140/
sentry: https://sentry.ceph.com/organizations/ceph/?query=bc5571e5b64a410aa707d3c7091fd994

    Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c
    ceph_test_lazy_omap_stats'

[7446200]  rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados tasks/rados_cls_all validater/valgrind}
-----------------------------------------------------------------
time:   01:45:38
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446200/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446200/

    saw valgrind issues

[7446207]  rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/connectivity task/test_extra_daemon_features}
-----------------------------------------------------------------
time:   00:19:46
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446207/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446207/

    SELinux denials found on ubuntu@smithi031.front.sepia.ceph.com: ['type=AVC
    msg=audit(1699094072.281:21965): avc:  denied  { ioctl } for  pid=125244
    comm="iptables" path="/var/lib/containers/storage/overlay/c324b2e1cdb2db590
    ae5e9d0b4bcf87178fe5194488b37099d17e7508c4a5683/merged" dev="overlay"
    ino=2228302 scontext=system_u:system_r:iptables_t:s0
    tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir
    permissive=1']

[7446210]  rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/scrub}
-----------------------------------------------------------------
time:   00:20:25
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446210/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446210/
sentry: https://sentry.ceph.com/organizations/ceph/?query=be2ee29780d749f68be64d8b4702195d

    Command failed (workunit test scrub/osd-recovery-scrub.sh) on smithi152
    with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd
    -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=23776148d05fd5bbb0eae6dde0ebb81fe2ceda46
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-
    scrub.sh'

[7446278]  rados/upgrade/parallel/{0-random-distro$/{rhel_8.6_container_tools_3.0} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}}
-----------------------------------------------------------------
time:   04:12:32
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446278/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446278/
sentry: https://sentry.ceph.com/organizations/ceph/?query=3aa4e876c0914e36b3f5dd86ca22b3b2

    Command failed (workunit test cls/test_cls_2pc_queue.sh) on smithi082 with
    status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=quincy TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph"
    CEPH_ID="0" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/cls/test_cls_2pc_queue.sh'

[7446298]  rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/snaps-few-objects}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446298/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446298/
sentry: https://sentry.ceph.com/organizations/ceph/?query=308185ef991943debbc4af56e063e798

    Command failed on smithi159 with status 100: 'sudo apt-get clean'

[7446308]  rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/classic task/test_orch_cli_mon}
-----------------------------------------------------------------
time:   00:13:04
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446308/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446308/
sentry: https://sentry.ceph.com/organizations/ceph/?query=30a71c9b30814c5ba1bfe90cd1d8f445

    No module named 'tasks.cephadm'

[7446314]  rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites}
-----------------------------------------------------------------
time:   00:10:04
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446314/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446314/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b3af845db8894e93b3dc44d045a49c17

    No module named 'tasks.ceph'

[7446318]  rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait}
-----------------------------------------------------------------
time:   00:08:55
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446318/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446318/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0d2c6227b3a345a88ec62f967e42e815

    Config file not found: "/home/teuthworker/src/github.com_ceph_ceph-
    c_23776148d05fd5bbb0eae6dde0ebb81fe2ceda46/qa/tasks/cephadm.conf".

[7446338]  rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:10:08
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446338/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446338/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5e012ee27f9f459dbc87353ae7830f1a

    No module named 'tasks'

[7446346]  rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind}
-----------------------------------------------------------------
time:   00:48:35
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446346/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446346/

    saw valgrind issues

[7446417]  rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-low-osd-mem-target} tasks/e2e}
-----------------------------------------------------------------
time:   00:30:47
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446417/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446417/
sentry: https://sentry.ceph.com/organizations/ceph/?query=86a2d1a83ab34d50a79199748a1c4b6c

    Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi057
    with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd
    -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=23776148d05fd5bbb0eae6dde0ebb81fe2ceda46
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh'

[7446445]  rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:11:34
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446445/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446445/
sentry: https://sentry.ceph.com/organizations/ceph/?query=92289b30088347658b2c8ab94338ece9

    Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c
    ceph_test_lazy_omap_stats'

[7446467]  rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest}
-----------------------------------------------------------------
time:   00:22:16
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446467/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446467/

    saw valgrind issues

[7446469]  rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind}
-----------------------------------------------------------------
time:   00:30:23
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446469/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446469/

    saw valgrind issues

[7446562]  rados/upgrade/parallel/{0-random-distro$/{rhel_8.6_container_tools_3.0} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}}
-----------------------------------------------------------------
time:   00:14:16
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446562/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446562/
sentry: https://sentry.ceph.com/organizations/ceph/?query=fbf117f1e37c44c0ac4c501c2baf24b1

    Command failed on smithi164 with status 127: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy
    shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    ff5e4048-7b21-11ee-8db9-212e2dc638e7 -- ceph mon dump -f json'

[7446578]  rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind}
-----------------------------------------------------------------
time:   03:48:56
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446578/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446578/
sentry: https://sentry.ceph.com/organizations/ceph/?query=6e9bb9e7ee89418c814dcd6545b233a1

    Command failed (workunit test cls/test_cls_lock.sh) on smithi133 with
    status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=23776148d05fd5bbb0eae6dde0ebb81fe2ceda46
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_lock.sh'

[7446660]  rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-zlib} tasks/e2e}
-----------------------------------------------------------------
time:   00:29:17
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446660/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446660/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4da64beb4b984c4a9c4c3f51cc23ab30

    Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi079
    with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd
    -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=23776148d05fd5bbb0eae6dde0ebb81fe2ceda46
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh'



Dead
=================================================================
[7446274]  rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{centos_latest}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446274/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446274/

    hit max job timeout

[7446296]  rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0} 1-start 2-services/basic 3-final}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446296/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446296/

    Error reimaging machines: Failed to power on smithi125

[7446306]  rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446306/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446306/

    Error reimaging machines: Failed to power on smithi125

[7446312]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/radosbench}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446312/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446312/

    Error reimaging machines: Failed to power on smithi125

[7446328]  rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/connectivity task/test_rgw_multisite}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446328/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446328/

    Error reimaging machines: Failed to power on smithi110

[7446342]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/redirect_promote_tests}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446342/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446342/

    Error reimaging machines: Failed to power on smithi110

[7446348]  rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/rados_python}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446348/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446348/

    Error reimaging machines: Failed to power on smithi125

[7446354]  rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/classic task/test_set_mon_crush_locations}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446354/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446354/

    Error reimaging machines: Failed to power on smithi110

[7446358]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/redirect_set_object}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446358/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446358/

    Error reimaging machines: Failed to power on smithi125

[7446370]  rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446370/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446370/

    Error reimaging machines: Failed to power on smithi125

[7446380]  rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446380/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446380/

    Error reimaging machines: Failed to power on smithi125

[7446394]  rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446394/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446394/

    Error reimaging machines: Failed to power on smithi110

[7446410]  rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446410/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446410/

    Error reimaging machines: reached maximum tries (101) after waiting for 600
    seconds

[7446558]  rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446558/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446558/

    hit max job timeout

[7446582]  rados/objectstore/{backends/objectstore-bluestore-a supported-random-distro$/{centos_latest}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446582/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446582/

    hit max job timeout

[7446583]  rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/cache-snaps}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446583/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446583/

    Error reimaging machines: reached maximum tries (101) after waiting for 600
    seconds



Pass
=================================================================
[7446130] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/classic task/test_ca_signed_key}
time:   00:15:14
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446130/

[7446132] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-snaps-few-objects-overwrites}
time:   00:28:28
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446132/

[7446134] rados/singleton/{all/deduptool mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest}}
time:   00:10:52
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446134/

[7446136] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados tasks/rados_api_tests validater/lockdep}
time:   00:25:09
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446136/

[7446138] rados/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_20.04} 1-start 2-services/rgw 3-final}
time:   00:19:45
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446138/

[7446142] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:12:24
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446142/

[7446144] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/cache-pool-snaps}
time:   00:22:10
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446144/

[7446146] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd}
time:   02:40:03
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446146/

[7446148] rados/singleton/{all/divergent_priors mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}}
time:   00:12:45
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446148/

[7446150] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/connectivity start}
time:   00:19:45
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446150/

[7446152] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} tasks/scrub_test}
time:   00:11:24
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446152/

[7446154] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{ubuntu_latest} tasks/prometheus}
time:   00:15:06
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446154/

[7446156] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/force-sync-many workloads/rados_mon_osdmap_prune}
time:   00:42:33
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446156/

[7446158] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}
time:   00:14:36
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446158/

[7446160] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/connectivity task/test_cephadm}
time:   00:17:00
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446160/

[7446162] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/cache-snaps-balanced}
time:   00:24:16
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446162/

[7446164] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_read}
time:   00:13:45
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446164/

[7446166] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/snaps-few-objects}
time:   00:33:46
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446166/

[7446168] rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}}
time:   00:13:35
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446168/

[7446170] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} tasks/mon_recovery}
time:   00:13:23
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446170/

[7446172] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2}
time:   00:26:33
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446172/

[7446174] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all}
time:   00:16:47
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446174/

[7446176] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{ubuntu_latest}}
time:   00:21:16
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446176/

[7446178] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/cache-snaps}
time:   00:22:07
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446178/

[7446180] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}}
time:   00:12:59
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446180/

[7446182] rados/objectstore/{backends/objectstore-memstore supported-random-distro$/{centos_latest}}
time:   00:08:02
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446182/

[7446184] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/classic task/test_cephadm_repos}
time:   00:11:23
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446184/

[7446186] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects}
time:   00:20:51
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446186/

[7446188] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}
time:   00:22:01
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446188/

[7446190] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:26:49
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446190/

[7446192] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/classic start}
time:   00:18:37
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446192/

[7446194] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache}
time:   00:13:46
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446194/

[7446196] rados/singleton/{all/ec-inconsistent-hinfo mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}}
time:   00:20:26
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446196/

[7446198] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} tasks/libcephsqlite}
time:   00:13:06
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446198/

[7446203] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_rw}
time:   00:14:27
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446203/

[7446204] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{centos_latest}}
time:   00:19:03
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446204/

[7446209] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:14:52
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446209/

[7446212] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{centos_latest} tasks/workunits}
time:   00:10:13
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446212/

[7446214] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}}
time:   00:35:36
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446214/

[7446216] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/dedup-io-mixed}
time:   00:11:48
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446216/

[7446218] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites}
time:   00:23:53
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446218/

[7446220] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add}
time:   00:19:08
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446220/

[7446222] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{centos_latest}}
time:   00:09:29
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446222/

[7446224] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_mon_workunits}
time:   00:32:50
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446224/

[7446226] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}}
time:   00:08:21
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446226/

[7446228] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}}
time:   00:17:49
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446228/

[7446230] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/dedup-io-snaps}
time:   00:16:43
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446230/

[7446232] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} tasks/mon_clock_no_skews}
time:   00:08:37
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446232/

[7446234] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2}
time:   00:29:02
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446234/

[7446236] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{centos_latest}}
time:   00:27:42
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446236/

[7446238] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start}
time:   00:24:24
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446238/

[7446240] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}}
time:   00:30:54
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446240/

[7446242] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write}
time:   00:14:01
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446242/

[7446244] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} tasks/rados_api_tests}
time:   00:27:11
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446244/

[7446246] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/pool-snaps-few-objects}
time:   00:28:49
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446246/

[7446248] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/test_rbd_api}
time:   00:21:57
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446248/

[7446250] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2}
time:   00:26:09
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446250/

[7446252] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{centos_latest}}
time:   00:12:07
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446252/

[7446254] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag}
time:   00:14:24
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446254/

[7446256] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}}
time:   00:34:37
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446256/

[7446258] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:26:53
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446258/

[7446260] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest}}
time:   00:30:31
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446260/

[7446262] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/connectivity task/test_orch_cli}
time:   00:16:21
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446262/

[7446264] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/rados_api_tests}
time:   00:29:41
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446264/

[7446266] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-zstd} tasks/dashboard}
time:   01:34:09
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446266/

[7446268] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-hybrid} supported-random-distro$/{centos_latest} tasks/crash}
time:   00:12:17
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446268/

[7446270] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{ubuntu_latest}}
time:   00:42:01
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446270/

[7446272] rados/rest/{mgr-restful supported-random-distro$/{ubuntu_latest}}
time:   00:14:28
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446272/

[7446276] rados/standalone/{supported-random-distro$/{centos_latest} workloads/c2c}
time:   00:15:32
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446276/

[7446280] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest}
time:   00:17:14
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446280/

[7446282] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados tasks/mon_recovery validater/lockdep}
time:   00:12:47
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446282/

[7446284] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:12:14
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446284/

[7446286] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest}}
time:   00:08:57
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446286/

[7446288] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:16:11
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446288/

[7446290] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/radosbench-high-concurrency}
time:   00:34:02
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446290/

[7446292] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{centos_latest}}
time:   00:10:36
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446292/

[7446294] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read}
time:   00:15:35
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446294/

[7446300] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}}
time:   00:14:31
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446300/

[7446302] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/rados_cls_all}
time:   00:22:15
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446302/

[7446304] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} tasks/mon_clock_with_skews}
time:   00:08:27
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446304/

[7446310] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{centos_latest}}
time:   00:11:32
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446310/

[7446316] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest}}
time:   00:14:11
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446316/

[7446320] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{centos_latest}}
time:   00:09:52
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446320/

[7446322] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/redirect}
time:   00:15:11
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446322/

[7446324] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1}
time:   00:24:12
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446324/

[7446326] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}}
time:   00:13:37
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446326/

[7446330] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:29:54
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446330/

[7446332] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-stupid} supported-random-distro$/{centos_latest} tasks/failover}
time:   00:15:20
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446332/

[7446334] rados/standalone/{supported-random-distro$/{centos_latest} workloads/crush}
time:   00:21:01
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446334/

[7446336] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read}
time:   00:14:19
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446336/

[7446340] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:15:25
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446340/

[7446344] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/cache-snaps}
time:   00:30:04
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446344/

[7446350] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}}
time:   00:11:25
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446350/

[7446352] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:14:40
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446352/

[7446356] rados/singleton-nomsgr/{all/ceph-snapmapper mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}
time:   00:11:10
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446356/

[7446360] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}}
time:   00:16:47
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446360/

[7446362] rados/objectstore/{backends/fusestore supported-random-distro$/{ubuntu_latest}}
time:   00:10:55
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446362/

[7446364] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/sync-many workloads/pool-create-delete}
time:   00:18:08
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446364/

[7446366] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate}
time:   00:19:10
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446366/

[7446368] rados/singleton-nomsgr/{all/crushdiff mon_election/classic rados supported-random-distro$/{ubuntu_latest}}
time:   00:11:50
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446368/

[7446372] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2}
time:   00:28:54
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446372/

[7446374] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/connectivity task/test_adoption}
time:   00:15:21
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446374/

[7446376] rados/singleton/{all/mon-config mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}}
time:   00:12:33
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446376/

[7446378] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/set-chunks-read}
time:   00:16:28
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446378/

[7446382] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}
time:   00:10:55
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446382/

[7446384] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:19:11
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446384/

[7446386] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-stupid} supported-random-distro$/{ubuntu_latest} tasks/insights}
time:   00:16:29
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446386/

[7446388] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/rados_stress_watch}
time:   00:22:32
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446388/

[7446390] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1}
time:   00:27:16
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446390/

[7446392] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}}
time:   00:20:29
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446392/

[7446396] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/small-objects-balanced}
time:   00:22:31
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446396/

[7446398] rados/standalone/{supported-random-distro$/{centos_latest} workloads/erasure-code}
time:   01:21:33
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446398/

[7446400] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:23:51
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446400/

[7446402] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/classic task/test_ca_signed_key}
time:   00:16:07
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446402/

[7446404] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{centos_latest}}
time:   00:09:25
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446404/

[7446405] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest}}
time:   00:31:35
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446405/

[7446406] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest}}
time:   00:19:54
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446406/

[7446407] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep}
time:   00:17:36
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446407/

[7446408] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all}
time:   00:18:16
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446408/

[7446409] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/small-objects-localized}
time:   00:19:32
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446409/

[7446411] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{centos_latest}}
time:   00:10:37
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446411/

[7446412] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read}
time:   00:17:00
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446412/

[7446413] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:18:26
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446413/

[7446415] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest}}
time:   00:12:42
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446415/

[7446419] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/radosbench}
time:   01:14:55
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446419/

[7446421] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/small-objects}
time:   00:22:59
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446421/

[7446423] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/rados_5925}
time:   00:13:16
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446423/

[7446425] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{centos_latest}}
time:   00:09:38
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446425/

[7446427] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/connectivity task/test_cephadm}
time:   00:18:41
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446427/

[7446429] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} tasks/rados_striper}
time:   00:08:57
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446429/

[7446431] rados/singleton/{all/peer mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest}}
time:   00:09:08
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446431/

[7446433] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews}
time:   00:11:04
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446433/

[7446435] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2}
time:   00:27:56
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446435/

[7446437] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{centos_latest} tasks/module_selftest}
time:   00:30:02
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446437/

[7446439] rados/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_20.04} 1-start 2-services/rgw 3-final}
time:   00:19:09
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446439/

[7446441] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/snaps-few-objects-balanced}
time:   00:27:12
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446441/

[7446443] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{centos_latest}}
time:   00:13:18
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446443/

[7446447] rados/standalone/{supported-random-distro$/{centos_latest} workloads/mgr}
time:   00:15:40
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446447/

[7446449] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}}
time:   00:19:34
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446449/

[7446451] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/classic task/test_cephadm_repos}
time:   00:09:27
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446451/

[7446453] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-radosbench}
time:   01:46:13
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446453/

[7446455] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write}
time:   00:13:39
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446455/

[7446457] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:24:59
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446457/

[7446459] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/snaps-few-objects-localized}
time:   00:28:09
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446459/

[7446461] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{centos_latest}}
time:   00:12:45
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446461/

[7446463] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add}
time:   00:16:47
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446463/

[7446465] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}}
time:   00:16:25
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446465/

[7446471] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites}
time:   00:33:01
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446471/

[7446473] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:12:21
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446473/

[7446475] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:23:49
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446475/

[7446477] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{centos_latest}}
time:   00:15:04
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446477/

[7446479] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/snaps-few-objects}
time:   00:27:40
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446479/

[7446481] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} tasks/rados_workunit_loadgen_big}
time:   00:30:01
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446481/

[7446483] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}}
time:   00:11:23
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446483/

[7446485] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/connectivity task/test_extra_daemon_features}
time:   00:21:24
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446485/

[7446487] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{centos_latest}}
time:   00:18:55
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446487/

[7446489] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} thrashers/force-sync-many workloads/rados_api_tests}
time:   00:27:25
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446489/

[7446491] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/write_fadvise_dontneed}
time:   00:17:08
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446491/

[7446493] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{centos_latest} tasks/per_module_finisher_stats}
time:   00:10:09
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446493/

[7446495] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write}
time:   00:23:54
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446495/

[7446497] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag}
time:   00:18:51
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446497/

[7446499] rados/singleton/{all/radostool mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest}}
time:   00:18:08
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446499/

[7446501] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} tasks/mon_clock_with_skews}
time:   00:08:44
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446501/

[7446502] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}
time:   00:12:15
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446502/

[7446503] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{centos_latest}}
time:   00:19:10
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446503/

[7446504] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/rbd_cls}
time:   00:21:50
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446504/

[7446505] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}}
time:   00:16:23
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446505/

[7446506] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/misc}
time:   01:14:22
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446506/

[7446507] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/admin_socket_objecter_requests}
time:   00:16:40
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446507/

[7446508] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest}}
time:   00:33:10
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446508/

[7446509] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-small-objects-balanced}
time:   00:20:44
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446509/

[7446511] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:16:44
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446511/

[7446513] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:23:49
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446513/

[7446515] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{ubuntu_latest}}
time:   00:11:36
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446515/

[7446517] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} tasks/rados_workunit_loadgen_mix}
time:   00:20:01
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446517/

[7446519] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{ubuntu_latest}}
time:   00:38:58
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446519/

[7446521] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-agent-big}
time:   00:19:10
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446521/

[7446523] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest}}
time:   00:13:21
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446523/

[7446524] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/connectivity task/test_orch_cli}
time:   00:18:45
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446524/

[7446526] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep}
time:   00:26:49
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446526/

[7446528] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:12:04
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446528/

[7446530] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{centos_latest}}
time:   00:25:40
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446530/

[7446532] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio}
time:   00:15:07
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446532/

[7446533] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait}
time:   00:14:20
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446533/

[7446534] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest}}
time:   00:15:57
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446534/

[7446535] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-agent-small}
time:   00:14:48
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446535/

[7446536] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}}
time:   00:35:17
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446536/

[7446537] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-bitmap} supported-random-distro$/{ubuntu_latest} tasks/progress}
time:   00:21:31
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446537/

[7446538] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{centos_latest}}
time:   00:11:59
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446538/

[7446539] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:15:59
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446539/

[7446540] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites}
time:   00:20:20
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446540/

[7446542] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} thrashers/many workloads/rados_mon_osdmap_prune}
time:   00:54:29
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446542/

[7446544] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest}}
time:   00:10:05
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446544/

[7446546] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy}
time:   00:23:42
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446546/

[7446548] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/classic task/test_orch_cli_mon}
time:   00:26:03
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446548/

[7446550] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} tasks/mon_recovery}
time:   00:15:11
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446550/

[7446552] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2}
time:   00:28:06
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446552/

[7446554] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} tasks/rados_workunit_loadgen_mostlyread}
time:   00:25:44
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446554/

[7446556] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-hybrid} tasks/dashboard}
time:   01:34:08
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446556/

[7446560] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/mon-stretch}
time:   00:26:17
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446560/

[7446564] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}}
time:   00:12:45
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446564/

[7446565] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/basic 3-final}
time:   00:13:32
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446565/

[7446566] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps}
time:   00:28:33
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446566/

[7446567] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench}
time:   00:13:16
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446567/

[7446568] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{ubuntu_latest}}
time:   00:12:15
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446568/

[7446569] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-fast-read}
time:   00:23:21
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446569/

[7446570] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:24:47
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446570/

[7446571] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/connectivity task/test_rgw_multisite}
time:   00:18:01
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446571/

[7446572] rados/singleton/{all/test-noautoscale-flag mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}}
time:   00:11:40
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446572/

[7446573] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/quincy backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/snaps-few-objects}
time:   00:37:02
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446573/

[7446574] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/cache-snaps-balanced}
time:   00:22:09
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446574/

[7446575] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rmdir-reactivate}
time:   00:14:43
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446575/

[7446576] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}
time:   00:13:45
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446576/

[7446577] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:11:59
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446577/

[7446579] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{centos_latest} tasks/prometheus}
time:   00:12:43
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446579/

[7446580] rados/singleton/{all/thrash-backfill-full mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}}
time:   02:21:59
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446580/

[7446581] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:19:43
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446581/

[7446584] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} tasks/readwrite}
time:   00:12:16
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446584/

[7446585] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{centos_latest}}
time:   00:09:22
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446585/

[7446586] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read}
time:   00:15:19
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446586/

[7446587] rados/singleton/{all/thrash-eio mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest}}
time:   00:44:31
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446587/

[7446588] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/classic task/test_set_mon_crush_locations}
time:   00:23:41
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446588/

[7446589] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/one workloads/rados_mon_workunits}
time:   00:28:17
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446589/

[7446590] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/mon}
time:   01:08:25
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446590/

[7446591] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/cache}
time:   00:11:45
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446591/

[7446592] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}
time:   00:10:24
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446592/

[7446593] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} tasks/mon_clock_no_skews}
time:   00:08:58
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446593/

[7446594] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2}
time:   00:25:05
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446594/

[7446595] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/connectivity task/test_adoption}
time:   00:16:16
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446595/

[7446596] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest}}
time:   00:20:43
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446596/

[7446597] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites}
time:   00:20:37
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446597/

[7446598] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/repave-all}
time:   00:13:49
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446598/

[7446599] rados/singleton-nomsgr/{all/ceph-snapmapper mon_election/classic rados supported-random-distro$/{ubuntu_latest}}
time:   00:10:56
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446599/

[7446600] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/dedup-io-mixed}
time:   00:12:16
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446600/

[7446601] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:28:19
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446601/

[7446602] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-many-deletes}
time:   00:22:43
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446602/

[7446603] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest}}
time:   00:20:02
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446603/

[7446604] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:18:22
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446604/

[7446605] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} tasks/repair_test}
time:   00:16:55
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446605/

[7446606] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw}
time:   00:14:51
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446606/

[7446607] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_latest} tasks/workunits}
time:   00:12:08
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446607/

[7446608] rados/singleton-nomsgr/{all/crushdiff mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}
time:   00:11:38
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446608/

[7446609] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/dedup-io-snaps}
time:   00:16:17
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446609/

[7446610] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:14:42
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446610/

[7446611] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest}
time:   00:17:27
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446611/

[7446612] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep}
time:   00:12:48
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446612/

[7446613] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/classic task/test_ca_signed_key}
time:   00:18:02
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446613/

[7446614] rados/singleton/{all/watch-notify-same-primary mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}}
time:   00:11:54
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446614/

[7446615] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/test_rbd_api}
time:   00:20:31
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446615/

[7446616] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{centos_latest}}
time:   00:09:11
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446616/

[7446617] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:25:19
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446617/

[7446618] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/pool-snaps-few-objects}
time:   00:27:26
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446618/

[7446620] rados/singleton/{all/admin-socket mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest}}
time:   00:09:08
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446620/

[7446622] rados/standalone/{supported-random-distro$/{centos_latest} workloads/osd-backfill}
time:   02:38:02
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446622/

[7446624] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}}
time:   00:35:19
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446624/

[7446626] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/sync-many workloads/pool-create-delete}
time:   00:17:22
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446626/

[7446628] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add}
time:   00:18:56
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446628/

[7446630] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{centos_latest}}
time:   00:09:28
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446630/

[7446632] rados/objectstore/{backends/objectstore-bluestore-b supported-random-distro$/{ubuntu_latest}}
time:   00:09:34
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446632/

[7446634] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews}
time:   00:10:27
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446634/

[7446636] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}
time:   00:14:14
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446636/

[7446638] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/rados_api_tests}
time:   00:29:22
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446638/

[7446640] rados/singleton/{all/backfill-toofull mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}}
time:   00:22:26
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446640/

[7446642] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/connectivity task/test_cephadm}
time:   00:18:34
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446642/

[7446644] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read}
time:   00:13:36
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446644/

[7446646] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} tasks/scrub_test}
time:   00:11:41
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446646/

[7446648] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{centos_latest}}
time:   00:10:25
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446648/

[7446650] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/classic task/test_cephadm_repos}
time:   00:10:36
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446650/

[7446652] rados/singleton/{all/deduptool mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}}
time:   00:13:21
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446652/

[7446654] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/radosbench-high-concurrency}
time:   00:23:15
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446654/

[7446656] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:28:29
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446656/

[7446658] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects}
time:   00:22:59
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446658/

[7446662] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity random-objectstore$/{bluestore-stupid} supported-random-distro$/{ubuntu_latest} tasks/crash}
time:   00:14:23
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446662/

[7446664] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites}
time:   00:27:20
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446664/

[7446666] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start}
time:   00:16:53
info:   http://pulpito.ceph.com/yuriw-2023-11-04_03:30:59-rados-main-distro-default-smithi/7446666/