2024-03-13T04:07:37.640 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-03-13T04:07:37.640 INFO:teuthology.results:subset: '111/120000' 2024-03-13T04:07:37.640 INFO:teuthology.results:seed: '3657' 2024-03-13T09:33:33.279 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-03-13T09:33:33.306 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2024-03-13T20:42:37.512 INFO:teuthology.results:Tests finished! gathering results... 2024-03-13T20:43:02.065 INFO:scrape:Found 304 jobs 2024-03-13T20:43:13.914 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594857/teuthology.log 2024-03-13T20:43:16.958 INFO:scrape:Found 14 distinct failure reasons 2024-03-13T20:43:16.959 INFO:scrape:Failure: 2024-03-13T20:43:16.959 INFO:scrape:5 jobs: ['7595194', '7594818', '7595124', '7595057', '7594958'] 2024-03-13T20:43:16.959 INFO:scrape:suites intersection: ['ceph', 'mon_election/classic', 'rados', 'rados/thrash-erasure-code-crush-4-nodes/{arch/x86_64', 'thrashosds-health', 'workloads/ec-rados-plugin=jerasure-k=8-m=6-crush}'] 2024-03-13T20:43:16.959 INFO:scrape:suites union: ['ceph', 'mon_election/classic', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/osd-delay', 'msgr-failures/osd-dispatch-delay', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-low-osd-mem-target', 'rados', 'rados/thrash-erasure-code-crush-4-nodes/{arch/x86_64', 'recovery-overrides/{more-active-recovery}', 'recovery-overrides/{more-async-partial-recovery}', 'recovery-overrides/{more-async-recovery}', 'supported-random-distro$/{centos_latest}', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/careful', 'thrashers/default', 'thrashers/mapgap', 'thrashers/morepggrow', 'thrashers/pggrow', 'thrashosds-health', 'workloads/ec-rados-plugin=jerasure-k=8-m=6-crush}'] 2024-03-13T20:43:16.960 INFO:scrape: 2024-03-13T20:43:16.960 INFO:scrape:Failure: Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_lazy_omap_stats' 2024-03-13T20:43:16.960 INFO:scrape:4 jobs: ['7595202', '7595150', '7595049', '7594978'] 2024-03-13T20:43:16.960 INFO:scrape:suites intersection: ['rados'] 2024-03-13T20:43:16.960 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados', 'rados/singleton-nomsgr/{all/ceph-snapmapper', 'rados/singleton-nomsgr/{all/lazy_omap_stats_output', 'supported-random-distro$/{centos_latest}}', 'supported-random-distro$/{ubuntu_latest}}'] 2024-03-13T20:43:16.960 INFO:scrape: 2024-03-13T20:43:16.960 INFO:scrape:Failure: Command failed on smithi139 with status 32: 'sync && sudo umount -f /var/lib/ceph/osd/ceph-5' 2024-03-13T20:43:16.960 INFO:scrape:2 jobs: ['7594970', '7594835'] 2024-03-13T20:43:16.960 INFO:scrape:suites intersection: [] 2024-03-13T20:43:16.961 INFO:scrape:suites union: ['agent/on', 'ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/few', 'msgr/async', 'objectstore/bluestore-bitmap', 'openstack}', 'rados', 'rados/cephadm/workunits/{0-distro/ubuntu_22.04', 'rados/verify/{centos_latest', 'task/test_ca_signed_key}', 'tasks/rados_api_tests', 'validater/valgrind}'] 2024-03-13T20:43:16.961 INFO:scrape: 2024-03-13T20:43:16.961 INFO:scrape:Timeout 120 running ceph --cluster ceph osd last-stat-seq osd.2 2024-03-13T20:43:16.961 INFO:scrape:1 jobs: ['7595139'] 2024-03-13T20:43:16.961 INFO:scrape:suites: ['mon_election/classic', 'msgr-failures/few', 'msgr/async', 'objectstore/bluestore-comp-zlib', 'rados', 'rados/singleton/{all/thrash-eio', 'supported-random-distro$/{centos_latest}}'] 2024-03-13T20:43:16.961 INFO:scrape: 2024-03-13T20:43:16.961 INFO:scrape:Dead: b'2024-03-13T17:22:20.825 DEBUG:teuthology.exit:Finished running handlers' 2024-03-13T20:43:16.961 INFO:scrape:2 jobs: ['7594897', '7595113'] 2024-03-13T20:43:16.961 INFO:scrape:suites intersection: ['rados', 'rados/singleton-nomsgr/{all/admin_socket_output', 'supported-random-distro$/{ubuntu_latest}}'] 2024-03-13T20:43:16.961 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados', 'rados/singleton-nomsgr/{all/admin_socket_output', 'supported-random-distro$/{ubuntu_latest}}'] 2024-03-13T20:43:16.961 INFO:scrape: 2024-03-13T20:43:16.962 INFO:scrape:Failure: valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun 2024-03-13T20:43:16.962 INFO:scrape:4 jobs: ['7595061', '7595129', '7594823', '7595060'] 2024-03-13T20:43:16.962 INFO:scrape:suites intersection: [] 2024-03-13T20:43:16.962 INFO:scrape:suites union: ['2-inject-leak/none', 'centos_latest}', 'ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async-v1only', 'msgr/async-v2only', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/valgrind-leaks/{1-start', 'rados/verify/{centos_latest', 'tasks/mon_recovery', 'tasks/rados_cls_all', 'validater/valgrind}'] 2024-03-13T20:43:16.962 INFO:scrape: 2024-03-13T20:43:16.962 INFO:scrape:Failure: failed to complete snap trimming before timeout 2024-03-13T20:43:16.962 INFO:scrape:2 jobs: ['7595115', '7594901'] 2024-03-13T20:43:16.962 INFO:scrape:suites intersection: ['0-start', '1-tasks', 'rados/upgrade/parallel/{0-random-distro$/{centos_9.stream}', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'test_rbd_api', 'test_rbd_python}}', 'upgrade-sequence', 'workload/{ec-rados-default'] 2024-03-13T20:43:16.962 INFO:scrape:suites union: ['0-start', '1-tasks', 'mon_election/classic', 'mon_election/connectivity', 'rados/upgrade/parallel/{0-random-distro$/{centos_9.stream}', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'test_rbd_api', 'test_rbd_python}}', 'upgrade-sequence', 'workload/{ec-rados-default'] 2024-03-13T20:43:16.963 INFO:scrape: 2024-03-13T20:43:16.963 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rados/test.sh 2024-03-13T20:43:16.963 INFO:scrape:2 jobs: ['7595099', '7594843'] 2024-03-13T20:43:16.963 INFO:scrape:suites intersection: ['ceph', 'clusters/{fixed-4', 'openstack}', 'rados', 'thrashosds-health', 'workloads/rados_api_tests}'] 2024-03-13T20:43:16.963 INFO:scrape:suites union: ['1-pg-log-overrides/normal_pg_log', '1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{default}', '2-recovery-overrides/{more-async-partial-recovery}', '3-scrub-overrides/{max-simultaneous-scrubs-1}', '3-scrub-overrides/{max-simultaneous-scrubs-3}', 'backoff/normal', 'backoff/peering_and_degraded', 'ceph', 'clusters/{fixed-4', 'crc-failures/bad_map_crc_failure', 'crc-failures/default', 'd-balancer/read', 'd-balancer/upmap-read', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/osd-dispatch-delay', 'msgr/async', 'msgr/async-v1only', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{centos_latest}', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/careful', 'thrashers/pggrow', 'thrashosds-health', 'workloads/rados_api_tests}'] 2024-03-13T20:43:16.963 INFO:scrape: 2024-03-13T20:43:16.963 INFO:scrape:Timeout 6h running clone.client.0/qa/workunits/rados/test.sh 2024-03-13T20:43:16.963 INFO:scrape:1 jobs: ['7595200'] 2024-03-13T20:43:16.963 INFO:scrape:suites: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_api_tests', 'validater/valgrind}'] 2024-03-13T20:43:16.963 INFO:scrape: 2024-03-13T20:43:16.964 INFO:scrape:Failure: "2024-03-13T08:00:00.000177+0000 mon.a (mon.0) 3355 : cluster 3 [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s)" in cluster log 2024-03-13T20:43:16.964 INFO:scrape:1 jobs: ['7594968'] 2024-03-13T20:43:16.964 INFO:scrape:suites: ['0-size-min-size-overrides/2-size-2-min-size', '1-install/nautilus-v1only', 'backoff/peering', 'ceph', 'clusters/{openstack', 'd-balancer/crush-compat', 'mon_election/classic', 'msgr-failures/few', 'rados', 'rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools}', 'thrashers/mapgap', 'thrashosds-health', 'three-plus-one}', 'workloads/radosbench}'] 2024-03-13T20:43:16.964 INFO:scrape: 2024-03-13T20:43:16.964 INFO:scrape:Failure: "2024-03-13T10:10:00.000290+0000 mon.a (mon.0) 2077 : cluster 3 [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull" in cluster log 2024-03-13T20:43:16.964 INFO:scrape:1 jobs: ['7595183'] 2024-03-13T20:43:16.964 INFO:scrape:suites: ['0-size-min-size-overrides/3-size-2-min-size', '1-install/quincy', 'backoff/normal', 'ceph', 'clusters/{openstack', 'd-balancer/on', 'mon_election/connectivity', 'msgr-failures/fastclose', 'rados', 'rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools}', 'thrashers/default', 'thrashosds-health', 'three-plus-one}', 'workloads/radosbench}'] 2024-03-13T20:43:16.964 INFO:scrape: 2024-03-13T20:43:16.964 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi154 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f85f9bfba5f90b21a3c8b0ccb8133e8de23a2501 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2024-03-13T20:43:16.964 INFO:scrape:1 jobs: ['7595036'] 2024-03-13T20:43:16.965 INFO:scrape:suites: ['debug/mgr', 'mon_election/connectivity', 'rados/dashboard/{0-single-container-host', 'random-objectstore$/{bluestore-stupid}', 'tasks/e2e}'] 2024-03-13T20:43:16.965 INFO:scrape: 2024-03-13T20:43:16.965 INFO:scrape:Dead: None 2024-03-13T20:43:16.965 INFO:scrape:1 jobs: ['7594857'] 2024-03-13T20:43:16.965 INFO:scrape: 2024-03-13T20:43:16.965 INFO:scrape:Failure: Command failed (workunit test scrub/osd-scrub-repair.sh) on smithi076 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f85f9bfba5f90b21a3c8b0ccb8133e8de23a2501 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh' 2024-03-13T20:43:16.965 INFO:scrape:1 jobs: ['7594833'] 2024-03-13T20:43:16.965 INFO:scrape:suites: ['rados/standalone/{supported-random-distro$/{centos_latest}', 'workloads/scrub}'] 2024-03-13T20:43:16.965 INFO:scrape: 2024-03-13T20:43:16.977 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi ================================================================= info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/ failed: 25 dead: 3 running: 0 waiting: 0 queued: 0 passed: 276 Fail ================================================================= [7594818] rados/thrash-erasure-code-crush-4-nodes/{arch/x86_64 ceph mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=8-m=6-crush} ----------------------------------------------------------------- time: 00:00:01 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594818/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594818/ sentry: https://sentry.ceph.com/organizations/ceph/?query=504163c326e8483091a673e0fe582623 [7594823] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados tasks/rados_cls_all validater/valgrind} ----------------------------------------------------------------- time: 01:47:32 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594823/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594823/ valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun [7594833] rados/standalone/{supported-random-distro$/{centos_latest} workloads/scrub} ----------------------------------------------------------------- time: 01:33:12 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594833/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594833/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e4fd21c79b484253ba570d0d459e093f Command failed (workunit test scrub/osd-scrub-repair.sh) on smithi076 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f85f9bfba5f90b21a3c8b0ccb8133e8de23a2501 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub- repair.sh' [7594835] rados/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_ca_signed_key} ----------------------------------------------------------------- time: 00:11:27 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594835/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594835/ sentry: https://sentry.ceph.com/organizations/ceph/?query=57b8020047a846a6a831af309d53781d Command failed on smithi089 with status 5: 'sudo systemctl stop ceph-571ef82e-e0f6-11ee-95c9-87774f69a715@mon.a' [7594843] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/rados_api_tests} ----------------------------------------------------------------- time: 03:15:59 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594843/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594843/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2931e17f791447b193296166e1f48e42 Command failed (workunit test rados/test.sh) on smithi122 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f85f9bfba5f90b21a3c8b0ccb8133e8de23a2501 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [7594901] rados/upgrade/parallel/{0-random-distro$/{centos_9.stream} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} ----------------------------------------------------------------- time: 00:41:19 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594901/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594901/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2b6b3683fde94936b350ba25966d52a4 failed to complete snap trimming before timeout [7594958] rados/thrash-erasure-code-crush-4-nodes/{arch/x86_64 ceph mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=8-m=6-crush} ----------------------------------------------------------------- time: 00:00:01 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594958/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594958/ sentry: https://sentry.ceph.com/organizations/ceph/?query=60858a3272bb4f1087728a66b7962343 [7594968] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/radosbench} ----------------------------------------------------------------- time: 01:53:54 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594968/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594968/ "2024-03-13T08:00:00.000177+0000 mon.a (mon.0) 3355 : cluster 3 [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s)" in cluster log [7594970] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} ----------------------------------------------------------------- time: 01:06:12 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594970/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594970/ Command failed on smithi139 with status 32: 'sync && sudo umount -f /var/lib/ceph/osd/ceph-5' [7594978] rados/singleton-nomsgr/{all/ceph-snapmapper mon_election/connectivity rados supported-random-distro$/{centos_latest}} ----------------------------------------------------------------- time: 00:07:58 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594978/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594978/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e71ce6f0076c4b4cabd108195b5e651c Command failed on smithi187 with status 1: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_snap_mapper' [7595036] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-stupid} tasks/e2e} ----------------------------------------------------------------- time: 00:19:09 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595036/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595036/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7259cb78662446338cb2ad2d41057347 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi154 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f85f9bfba5f90b21a3c8b0ccb8133e8de23a2501 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [7595049] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:10:00 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595049/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595049/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e293222db871461eb054539c85cbfbdc Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_lazy_omap_stats' [7595057] rados/thrash-erasure-code-crush-4-nodes/{arch/x86_64 ceph mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=8-m=6-crush} ----------------------------------------------------------------- time: 00:00:01 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595057/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595057/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c9ba4242dad64b678c1fd7ff58416896 [7595060] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} ----------------------------------------------------------------- time: 00:24:31 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595060/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595060/ valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun [7595061] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} ----------------------------------------------------------------- time: 00:31:18 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595061/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595061/ valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun [7595099] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/rados_api_tests} ----------------------------------------------------------------- time: 03:17:43 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595099/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595099/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c359b08a9e5b466495d80d460ab84b6b Command failed (workunit test rados/test.sh) on smithi129 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f85f9bfba5f90b21a3c8b0ccb8133e8de23a2501 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [7595115] rados/upgrade/parallel/{0-random-distro$/{centos_9.stream} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} ----------------------------------------------------------------- time: 00:41:22 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595115/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595115/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b0f25588aad84fff843c9ceb4b732c78 failed to complete snap trimming before timeout [7595124] rados/thrash-erasure-code-crush-4-nodes/{arch/x86_64 ceph mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=8-m=6-crush} ----------------------------------------------------------------- time: 00:00:01 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595124/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595124/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ed9e407eb6ed46d7983f039741ff3e11 [7595129] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} ----------------------------------------------------------------- time: 01:51:32 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595129/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595129/ valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun [7595139] rados/singleton/{all/thrash-eio mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest}} ----------------------------------------------------------------- time: 00:51:43 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595139/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595139/ Command failed on smithi062 with status 124: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2' [7595150] rados/singleton-nomsgr/{all/ceph-snapmapper mon_election/classic rados supported-random-distro$/{centos_latest}} ----------------------------------------------------------------- time: 00:07:56 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595150/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595150/ sentry: https://sentry.ceph.com/organizations/ceph/?query=aaa7f1f7b8aa49cebb2c7bc88d4afc63 Command failed on smithi032 with status 1: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_snap_mapper' [7595183] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/quincy backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/radosbench} ----------------------------------------------------------------- time: 01:17:37 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595183/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595183/ "2024-03-13T10:10:00.000290+0000 mon.a (mon.0) 2077 : cluster 3 [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull" in cluster log [7595194] rados/thrash-erasure-code-crush-4-nodes/{arch/x86_64 ceph mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=8-m=6-crush} ----------------------------------------------------------------- time: 00:00:01 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595194/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595194/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d848e2ab37fb49e2889579d17fbc8a8e [7595200] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} ----------------------------------------------------------------- time: 06:33:41 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595200/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595200/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d16b54d00c5d4cb5b4be1a208ccc30da Command failed (workunit test rados/test.sh) on smithi103 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f85f9bfba5f90b21a3c8b0ccb8133e8de23a2501 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 ALLOW_TIMEOUTS=1 adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [7595202] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{centos_latest}} ----------------------------------------------------------------- time: 00:07:34 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595202/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595202/ sentry: https://sentry.ceph.com/organizations/ceph/?query=eaed346ab9ee4037bdeb326c595d7e8c Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_lazy_omap_stats' Dead ================================================================= [7594857] rados/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_cephadm} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594857/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594857/ Error reimaging machines: Failed to power on smithi164 [7594897] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594897/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594897/ hit max job timeout [7595113] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595113/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595113/ hit max job timeout Pass ================================================================= [7594784] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:12:12 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594784/ [7594786] rados/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:24:47 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594786/ [7594788] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:13:02 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594788/ [7594790] rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:12:19 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594790/ [7594792] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache} time: 00:18:32 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594792/ [7594794] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:13:55 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594794/ [7594796] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:28:19 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594796/ [7594798] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:14:27 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594798/ [7594800] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/dedup-io-mixed} time: 00:17:13 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594800/ [7594802] rados/thrash-erasure-code/{ceph clusters/{fixed-4 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-radosbench} time: 01:09:22 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594802/ [7594804] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:11:51 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594804/ [7594806] rados/objectstore/{backends/objectstore-memstore supported-random-distro$/{ubuntu_latest}} time: 00:09:32 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594806/ [7594808] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/quincy backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/test_rbd_api} time: 00:24:48 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594808/ [7594810] rados/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:13:26 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594810/ [7594812] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:17:27 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594812/ [7594814] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/dedup-io-snaps} time: 00:15:52 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594814/ [7594815] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:28 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594815/ [7594820] rados/singleton/{all/ec-inconsistent-hinfo mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest}} time: 00:15:09 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594820/ [7594821] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} tasks/libcephsqlite} time: 00:12:07 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594821/ [7594825] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:13:18 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594825/ [7594828] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/pool-snaps-few-objects} time: 00:26:59 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594828/ [7594830] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:18:21 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594830/ [7594832] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:42 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594832/ [7594837] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{centos_latest} tasks/workunits} time: 00:09:12 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594837/ [7594839] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}} time: 00:37:06 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594839/ [7594841] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-4 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:44:44 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594841/ [7594845] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:09:20 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594845/ [7594847] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} thrashers/many workloads/rados_mon_workunits} time: 00:29:02 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594847/ [7594849] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}} time: 00:07:26 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594849/ [7594851] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/radosbench-high-concurrency} time: 00:30:42 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594851/ [7594853] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} tasks/mon_clock_no_skews} time: 00:08:05 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594853/ [7594855] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:25:05 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594855/ [7594859] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:23:46 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594859/ [7594861] rados/thrash-erasure-code/{ceph clusters/{fixed-4 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/ec-small-objects-balanced} time: 00:21:17 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594861/ [7594863] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:31:10 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594863/ [7594865] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/radosbench} time: 01:35:03 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594865/ [7594867] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:13:51 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594867/ [7594869] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/rados_api_tests} time: 00:45:25 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594869/ [7594871] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:09:43 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594871/ [7594873] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}} time: 00:30:29 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594873/ [7594875] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/redirect} time: 00:15:45 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594875/ [7594877] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:38 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594877/ [7594879] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest}} time: 00:27:45 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594879/ [7594881] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/reef backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/cache-snaps} time: 00:34:31 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594881/ [7594883] rados/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:14:00 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594883/ [7594885] rados/thrash-erasure-code-crush-4-nodes/{arch/x86_64 ceph mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=2-crush} time: 00:21:38 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594885/ [7594888] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-stupid} tasks/dashboard} time: 01:08:55 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594888/ [7594889] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:12:47 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594889/ [7594891] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{centos_latest}} time: 00:35:49 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594891/ [7594894] rados/rest/{mgr-restful supported-random-distro$/{centos_latest}} time: 00:09:49 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594894/ [7594899] rados/standalone/{supported-random-distro$/{centos_latest} workloads/c2c} time: 00:13:53 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594899/ [7594903] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:17:16 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594903/ [7594905] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados tasks/mon_recovery validater/lockdep} time: 00:12:41 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594905/ [7594907] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/redirect_promote_tests} time: 00:20:09 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594907/ [7594909] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:04 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594909/ [7594911] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:10:21 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594911/ [7594914] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/redirect_set_object} time: 00:12:18 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594914/ [7594915] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:09:21 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594915/ [7594917] rados/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:14:38 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594917/ [7594920] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:13:50 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594920/ [7594922] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/snaps-few-objects} time: 00:37:05 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594922/ [7594924] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest}} time: 00:11:53 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594924/ [7594926] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} tasks/rados_cls_all} time: 00:15:23 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594926/ [7594928] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/set-chunks-read} time: 00:18:22 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594928/ [7594930] rados/thrash-erasure-code/{ceph clusters/{fixed-4 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-small-objects-fast-read} time: 00:20:39 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594930/ [7594932] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:10:17 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594932/ [7594934] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:14:36 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594934/ [7594936] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:11:05 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594936/ [7594938] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-4 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:24:26 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594938/ [7594940] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:14:40 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594940/ [7594942] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/small-objects-balanced} time: 00:20:44 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594942/ [7594944] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream_runc} 1-start 2-services/basic 3-final} time: 00:11:09 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594944/ [7594946] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:10:52 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594946/ [7594948] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/small-objects-localized} time: 00:21:54 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594948/ [7594950] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:12:41 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594950/ [7594952] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:31 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594952/ [7594954] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{centos_latest} tasks/failover} time: 00:13:39 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594954/ [7594956] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/crush} time: 00:23:02 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594956/ [7594960] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:14:02 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594960/ [7594962] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:07:56 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594962/ [7594964] rados/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_cephadm_repos} time: 00:05:56 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594964/ [7594966] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/small-objects} time: 00:22:23 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594966/ [7594972] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} tasks/rados_python} time: 00:14:33 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594972/ [7594974] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:12:47 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594974/ [7594976] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:14:12 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594976/ [7594980] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-balanced} time: 00:27:33 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594980/ [7594982] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:16:34 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594982/ [7594984] rados/objectstore/{backends/fusestore supported-random-distro$/{centos_latest}} time: 00:08:02 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594984/ [7594986] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/sync-many workloads/pool-create-delete} time: 00:19:32 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594986/ [7594988] rados/thrash-erasure-code/{ceph clusters/{fixed-4 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:21:46 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594988/ [7594990] rados/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:19:32 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594990/ [7594992] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/snaps-few-objects-localized} time: 00:25:46 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594992/ [7594994] rados/singleton-nomsgr/{all/crushdiff mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:08:56 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594994/ [7594996] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} tasks/mon_recovery} time: 00:23:27 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594996/ [7594998] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:25:17 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7594998/ [7595000] rados/singleton/{all/mon-config mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest}} time: 00:13:30 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595000/ [7595001] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:14:06 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595001/ [7595004] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/snaps-few-objects} time: 00:29:50 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595004/ [7595005] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:07:58 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595005/ [7595007] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-hybrid} supported-random-distro$/{centos_latest} tasks/insights} time: 00:11:19 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595007/ [7595009] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} tasks/rados_stress_watch} time: 00:14:27 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595009/ [7595011] rados/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_extra_daemon_features} time: 00:18:57 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595011/ [7595013] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:19:39 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595013/ [7595016] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-4 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:23:48 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595016/ [7595018] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/write_fadvise_dontneed} time: 00:17:09 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595018/ [7595019] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/erasure-code} time: 01:38:22 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595019/ [7595021] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:04 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595021/ [7595023] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:10:36 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595023/ [7595024] rados/thrash-erasure-code-crush-4-nodes/{arch/x86_64 ceph mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=2-crush} time: 00:19:30 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595024/ [7595025] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest}} time: 00:30:56 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595025/ [7595026] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/admin_socket_objecter_requests} time: 00:20:39 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595026/ [7595027] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:22:18 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595027/ [7595028] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep} time: 00:17:36 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595028/ [7595029] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:17 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595029/ [7595030] rados/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:14:56 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595030/ [7595031] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/rbd_cls} time: 00:24:21 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595031/ [7595032] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:11:48 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595032/ [7595033] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:14:23 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595033/ [7595034] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/cache-agent-big} time: 00:15:28 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595034/ [7595035] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest}} time: 00:12:23 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595035/ [7595037] rados/thrash-erasure-code/{ceph clusters/{fixed-4 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects} time: 00:20:20 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595037/ [7595038] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} thrashers/sync workloads/rados_5925} time: 00:10:18 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595038/ [7595039] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/cache-agent-small} time: 00:16:00 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595039/ [7595040] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:11:06 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595040/ [7595041] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/rados_striper} time: 00:11:32 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595041/ [7595042] rados/singleton/{all/peer mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:10:46 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595042/ [7595043] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:09:58 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595043/ [7595044] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:26:49 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595044/ [7595045] rados/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_host_drain} time: 00:15:35 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595045/ [7595046] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-stupid} supported-random-distro$/{ubuntu_latest} tasks/module_selftest} time: 00:25:36 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595046/ [7595047] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:52:07 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595047/ [7595048] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{ubuntu_latest}} time: 00:16:21 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595048/ [7595050] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/mgr} time: 00:17:29 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595050/ [7595051] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:18:28 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595051/ [7595052] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps} time: 00:23:51 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595052/ [7595053] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:12:08 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595053/ [7595054] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:12 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595054/ [7595055] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:10:01 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595055/ [7595056] rados/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:15:44 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595056/ [7595058] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:18:05 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595058/ [7595059] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-snaps-balanced} time: 00:23:32 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595059/ [7595062] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-4 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:32:53 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595062/ [7595063] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:45 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595063/ [7595064] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:15:16 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595064/ [7595065] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} tasks/rados_workunit_loadgen_big} time: 00:30:38 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595065/ [7595066] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/cache-snaps} time: 00:25:12 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595066/ [7595067] rados/thrash-erasure-code/{ceph clusters/{fixed-4 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:45:41 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595067/ [7595068] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:10:19 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595068/ [7595069] rados/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_iscsi_container/{centos_9.stream test_iscsi_container}} time: 00:13:27 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595069/ [7595070] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/snaps-few-objects} time: 00:35:29 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595070/ [7595071] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:22:21 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595071/ [7595072] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} thrashers/force-sync-many workloads/rados_api_tests} time: 00:27:11 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595072/ [7595073] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/cache} time: 00:15:14 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595073/ [7595074] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-hybrid} supported-random-distro$/{ubuntu_latest} tasks/per_module_finisher_stats} time: 00:11:33 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595074/ [7595075] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:23:11 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595075/ [7595076] rados/singleton/{all/radostool mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:20:31 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595076/ [7595077] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} tasks/mon_clock_with_skews} time: 00:08:00 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595077/ [7595078] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:11:08 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595078/ [7595079] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/dedup-io-mixed} time: 00:12:00 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595079/ [7595080] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:18:41 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595080/ [7595081] rados/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/repave-all} time: 00:13:27 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595081/ [7595082] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/misc} time: 01:11:10 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595082/ [7595083] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:27:21 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595083/ [7595084] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/dedup-io-snaps} time: 00:14:41 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595084/ [7595085] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:23 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595085/ [7595086] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:10:33 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595086/ [7595087] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} tasks/rados_workunit_loadgen_mix} time: 00:19:26 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595087/ [7595088] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{centos_latest}} time: 00:23:22 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595088/ [7595089] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest}} time: 00:12:26 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595089/ [7595090] rados/thrash-erasure-code-crush-4-nodes/{arch/x86_64 ceph mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=2-crush} time: 00:21:40 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595090/ [7595091] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/pool-snaps-few-objects} time: 00:30:51 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595091/ [7595092] rados/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_monitoring_stack_basic} time: 00:25:07 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595092/ [7595093] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} time: 00:24:11 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595093/ [7595094] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:03 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595094/ [7595095] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:13:03 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595095/ [7595096] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:14:01 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595096/ [7595097] rados/thrash-erasure-code/{ceph clusters/{fixed-4 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:27:45 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595097/ [7595098] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest}} time: 00:12:31 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595098/ [7595100] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:33:47 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595100/ [7595101] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-hybrid} supported-random-distro$/{ubuntu_latest} tasks/progress} time: 00:21:30 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595101/ [7595102] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:11:36 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595102/ [7595103] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-4 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:23:39 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595103/ [7595104] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/radosbench-high-concurrency} time: 00:51:46 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595104/ [7595105] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} thrashers/many workloads/rados_mon_osdmap_prune} time: 00:53:37 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595105/ [7595106] rados/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:19:54 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595106/ [7595107] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/test_rbd_api} time: 00:24:05 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595107/ [7595108] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest}} time: 00:08:29 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595108/ [7595109] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:17:11 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595109/ [7595110] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:27:22 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595110/ [7595111] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} tasks/rados_workunit_loadgen_mostlyread} time: 00:24:24 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595111/ [7595112] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-comp-lz4} tasks/dashboard} time: 01:09:35 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595112/ [7595114] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/mon-stretch} time: 00:26:29 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595114/ [7595116] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/radosbench} time: 00:44:31 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595116/ [7595117] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest}} time: 00:10:24 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595117/ [7595118] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:12:32 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595118/ [7595119] rados/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_rgw_multisite} time: 00:13:31 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595119/ [7595120] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/redirect} time: 00:17:07 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595120/ [7595121] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:09:13 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595121/ [7595122] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:46 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595122/ [7595123] rados/singleton/{all/test-noautoscale-flag mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}} time: 00:09:11 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595123/ [7595125] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/redirect_promote_tests} time: 00:14:37 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595125/ [7595126] rados/thrash-erasure-code/{ceph clusters/{fixed-4 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:24:11 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595126/ [7595127] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:14:06 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595127/ [7595128] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:27 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595128/ [7595130] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{centos_latest} tasks/prometheus} time: 00:10:34 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595130/ [7595131] rados/singleton/{all/thrash-backfill-full mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 01:46:02 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595131/ [7595132] rados/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:24:38 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595132/ [7595133] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/redirect_set_object} time: 00:14:58 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595133/ [7595134] rados/objectstore/{backends/objectstore-bluestore-a supported-random-distro$/{ubuntu_latest}} time: 02:31:05 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595134/ [7595135] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/readwrite} time: 00:14:53 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595135/ [7595136] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:10:53 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595136/ [7595137] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:14:17 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595137/ [7595138] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/set-chunks-read} time: 00:18:40 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595138/ [7595140] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/rados_mon_workunits} time: 00:30:45 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595140/ [7595141] rados/standalone/{supported-random-distro$/{centos_latest} workloads/mon} time: 01:02:54 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595141/ [7595142] rados/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_22.04} 1-start 2-services/rgw 3-final} time: 00:19:33 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595142/ [7595143] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:10:09 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595143/ [7595144] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:10:26 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595144/ [7595145] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:27:23 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595145/ [7595146] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/cache-snaps} time: 00:33:46 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595146/ [7595147] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/small-objects-balanced} time: 00:23:56 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595147/ [7595148] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:22:15 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595148/ [7595149] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-4 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites} time: 00:24:08 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595149/ [7595151] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/small-objects-localized} time: 00:23:45 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595151/ [7595152] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:17 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595152/ [7595153] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest}} time: 00:18:56 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595153/ [7595154] rados/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_set_mon_crush_locations} time: 00:16:17 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595154/ [7595155] rados/thrash-erasure-code/{ceph clusters/{fixed-4 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=8-m=6-crush} time: 00:23:54 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595155/ [7595156] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/repair_test} time: 00:18:15 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595156/ [7595157] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/small-objects} time: 00:21:24 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595157/ [7595158] rados/thrash-erasure-code-crush-4-nodes/{arch/x86_64 ceph mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=2-crush} time: 00:19:10 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595158/ [7595159] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:14:22 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595159/ [7595160] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-bitmap} supported-random-distro$/{ubuntu_latest} tasks/workunits} time: 00:11:33 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595160/ [7595161] rados/singleton-nomsgr/{all/crushdiff mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:08:45 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595161/ [7595162] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:14:15 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595162/ [7595163] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:16:55 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595163/ [7595164] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} time: 00:11:44 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595164/ [7595165] rados/singleton/{all/watch-notify-same-primary mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest}} time: 00:08:35 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595165/ [7595166] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/snaps-few-objects-balanced} time: 00:25:06 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595166/ [7595167] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:10:13 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595167/ [7595168] rados/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:14:01 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595168/ [7595169] rados/singleton/{all/admin-socket mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:09:48 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595169/ [7595170] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/snaps-few-objects-localized} time: 00:28:04 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595170/ [7595171] rados/standalone/{supported-random-distro$/{centos_latest} workloads/osd-backfill} time: 02:25:57 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595171/ [7595172] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}} time: 00:31:09 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595172/ [7595173] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/sync-many workloads/pool-create-delete} time: 00:14:46 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595173/ [7595174] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:10:33 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595174/ [7595175] rados/objectstore/{backends/objectstore-bluestore-b supported-random-distro$/{ubuntu_latest}} time: 00:08:45 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595175/ [7595176] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} tasks/mon_clock_with_skews} time: 00:07:31 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595176/ [7595177] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:14:08 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595177/ [7595178] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/snaps-few-objects} time: 00:27:22 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595178/ [7595179] rados/singleton/{all/backfill-toofull mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest}} time: 00:18:55 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595179/ [7595180] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:13:05 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595180/ [7595181] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} tasks/scrub_test} time: 00:11:46 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595181/ [7595182] rados/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_ca_signed_key} time: 00:13:17 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595182/ [7595184] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:09:30 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595184/ [7595185] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/write_fadvise_dontneed} time: 00:14:40 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595185/ [7595186] rados/singleton/{all/deduptool mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}} time: 00:10:17 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595186/ [7595187] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:30:14 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595187/ [7595188] rados/thrash-erasure-code/{ceph clusters/{fixed-4 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-radosbench} time: 01:39:42 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595188/ [7595189] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-bitmap} tasks/e2e} time: 00:22:52 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595189/ [7595190] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity random-objectstore$/{bluestore-bitmap} supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:12:20 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595190/ [7595191] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-4 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:42:59 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595191/ [7595192] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/admin_socket_objecter_requests} time: 00:25:14 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595192/ [7595193] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:09:06 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595193/ [7595195] rados/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:14:13 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595195/ [7595196] rados/singleton/{all/divergent_priors mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}} time: 00:09:56 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595196/ [7595198] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:12 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595198/ [7595201] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/normal ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/cache-agent-big} time: 00:44:57 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595201/ [7595203] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:13:26 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595203/ [7595204] rados/singleton/{all/divergent_priors2 mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:11:52 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595204/ [7595205] rados/standalone/{supported-random-distro$/{centos_latest} workloads/osd} time: 02:28:31 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595205/ [7595206] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} tasks/libcephsqlite} time: 00:12:23 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595206/ [7595207] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-4 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache-agent-small} time: 00:14:58 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595207/ [7595208] rados/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:15:34 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595208/ [7595209] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/rados_5925} time: 00:12:48 info: https://pulpito.ceph.com/yuriw-2024-03-12_18:31:57-rados-wip-yuri7-testing-2024-03-11-1229-distro-default-smithi/7595209/