2023-10-11T02:55:30.612 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-11T02:55:30.613 INFO:teuthology.results:seed: '7571' 2023-10-11T03:27:49.334 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-11T03:27:49.338 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-10-11T06:26:03.329 INFO:teuthology.results:Tests finished! gathering results... 2023-10-11T06:26:05.946 INFO:scrape:Found 34 jobs 2023-10-11T06:26:07.730 INFO:scrape:Found 4 distinct failure reasons 2023-10-11T06:26:07.731 INFO:scrape:Failure: "2023-10-11T03:34:43.161846+0000 mon.a (mon.0) 152 : cluster [WRN] Health check failed: Degraded data redundancy: 21/1826 objects degraded (1.150%), 8 pgs degraded (PG_DEGRADED)" in cluster log 2023-10-11T06:26:07.731 INFO:scrape:6 jobs: ['7420981', '7420985', '7420977', '7420992', '7420975', '7420979'] 2023-10-11T06:26:07.732 INFO:scrape:suites intersection: ['crimson-rados/perf/{clusters/{fixed-2}', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/bluestore', 'settings/optimized'] 2023-10-11T06:26:07.732 INFO:scrape:suites union: ['crimson-rados/perf/{clusters/{fixed-2}', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/bluestore', 'settings/optimized', 'workloads/radosbench_4K_rand_read}', 'workloads/radosbench_4K_seq_read}', 'workloads/radosbench_4M_rand_read}', 'workloads/radosbench_4M_seq_read}', 'workloads/radosbench_4M_write}', 'workloads/sample_radosbench}'] 2023-10-11T06:26:07.733 INFO:scrape: 2023-10-11T06:26:07.733 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rbd/crimson/test_crimson_librbd.sh 2023-10-11T06:26:07.733 INFO:scrape:2 jobs: ['7420967', '7420961'] 2023-10-11T06:26:07.734 INFO:scrape:suites intersection: ['crimson-rados/rbd/{clusters/fixed-1', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph'] 2023-10-11T06:26:07.734 INFO:scrape:suites union: ['crimson-rados/rbd/{clusters/fixed-1', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'tasks/rbd_api_tests_old_format}', 'tasks/rbd_api_tests}'] 2023-10-11T06:26:07.735 INFO:scrape: 2023-10-11T06:26:07.735 INFO:scrape:Failure: Command failed on smithi170 with status 6: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats' 2023-10-11T06:26:07.735 INFO:scrape:2 jobs: ['7420986', '7420963'] 2023-10-11T06:26:07.736 INFO:scrape:suites intersection: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{default}', 'clusters/{fixed-2}', 'crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/bluestore', 'thrashers/default', 'thrashosds-health'] 2023-10-11T06:26:07.736 INFO:scrape:suites union: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{default}', 'clusters/{fixed-2}', 'crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/bluestore', 'thrashers/default', 'thrashosds-health', 'workloads/admin_socket_objecter_requests}', 'workloads/snaps-few-objects-localized}'] 2023-10-11T06:26:07.736 INFO:scrape: 2023-10-11T06:26:07.737 INFO:scrape:Failure: reached maximum tries (1551) after waiting for 9300 seconds 2023-10-11T06:26:07.737 INFO:scrape:1 jobs: ['7420987'] 2023-10-11T06:26:07.738 INFO:scrape:suites: ['crimson-rados/perf/{clusters/{fixed-2}', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/bluestore', 'settings/optimized', 'workloads/radosbench_omap_write}'] 2023-10-11T06:26:07.738 INFO:scrape: 2023-10-11T06:26:07.740 INFO:teuthology.results:Sending results to yingxin.cheng@intel.com: Test Run: yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/ failed: 11 dead: 0 running: 0 waiting: 0 queued: 0 passed: 23 Fail ================================================================= [7420961] crimson-rados/rbd/{clusters/fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rbd_api_tests} ----------------------------------------------------------------- time: 03:12:24 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420961/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420961/ sentry: https://sentry.ceph.com/organizations/ceph/?query=36f400b0037e42c184bf7d210525e622 Command failed (workunit test rbd/crimson/test_crimson_librbd.sh) on smithi031 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=21780a4fb1b788d3aac31a2a2bfba73265c986e7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=61 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/crimson/test_crims on_librbd.sh' [7420963] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/admin_socket_objecter_requests} ----------------------------------------------------------------- time: 00:21:48 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420963/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420963/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3a805b4fdce146849294f4343ee7dfb1 Command failed on smithi006 with status 124: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json' [7420967] crimson-rados/rbd/{clusters/fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rbd_api_tests_old_format} ----------------------------------------------------------------- time: 03:12:09 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420967/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420967/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4e45df86ac904f0c80ec5554b4d10899 Command failed (workunit test rbd/crimson/test_crimson_librbd.sh) on smithi123 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=21780a4fb1b788d3aac31a2a2bfba73265c986e7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cep htest/clone.client.0/qa/workunits/rbd/crimson/test_crimson_librbd.sh' [7420975] crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/radosbench_4K_rand_read} ----------------------------------------------------------------- time: 00:12:46 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420975/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420975/ "2023-10-11T03:31:07.515752+0000 mon.a (mon.0) 151 : cluster [WRN] Health check failed: Degraded data redundancy: 121/11276 objects degraded (1.073%), 6 pgs degraded (PG_DEGRADED)" in cluster log [7420977] crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/radosbench_4K_seq_read} ----------------------------------------------------------------- time: 00:12:02 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420977/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420977/ "2023-10-11T03:33:07.144005+0000 mon.a (mon.0) 148 : cluster [WRN] Health check failed: Degraded data redundancy: 134/11572 objects degraded (1.158%), 7 pgs degraded (PG_DEGRADED)" in cluster log [7420979] crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/radosbench_4M_rand_read} ----------------------------------------------------------------- time: 00:13:02 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420979/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420979/ "2023-10-11T03:33:56.545993+0000 mon.a (mon.0) 152 : cluster [WRN] Health check failed: Degraded data redundancy: 32/1724 objects degraded (1.856%), 9 pgs degraded (PG_DEGRADED)" in cluster log [7420981] crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/radosbench_4M_seq_read} ----------------------------------------------------------------- time: 00:12:06 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420981/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420981/ "2023-10-11T03:34:43.161846+0000 mon.a (mon.0) 152 : cluster [WRN] Health check failed: Degraded data redundancy: 21/1826 objects degraded (1.150%), 8 pgs degraded (PG_DEGRADED)" in cluster log [7420985] crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/radosbench_4M_write} ----------------------------------------------------------------- time: 00:11:41 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420985/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420985/ "2023-10-11T03:40:49.497969+0000 mon.a (mon.0) 151 : cluster [WRN] Health check failed: Degraded data redundancy: 29/1530 objects degraded (1.895%), 6 pgs degraded (PG_DEGRADED)" in cluster log [7420986] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/snaps-few-objects-localized} ----------------------------------------------------------------- time: 00:27:36 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420986/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420986/ Command failed on smithi170 with status 6: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats' [7420987] crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/radosbench_omap_write} ----------------------------------------------------------------- time: 02:51:43 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420987/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420987/ reached maximum tries (1551) after waiting for 9300 seconds [7420992] crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/sample_radosbench} ----------------------------------------------------------------- time: 00:12:05 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420992/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420992/ "2023-10-11T03:47:53.246022+0000 mon.a (mon.0) 155 : cluster [WRN] Health check failed: Degraded data redundancy: 131/9796 objects degraded (1.337%), 7 pgs degraded (PG_DEGRADED)" in cluster log Pass ================================================================= [7420959] crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_api_tests} time: 00:26:37 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420959/ [7420960] crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/fio_4K_rand_read} time: 00:16:45 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420960/ [7420962] crimson-rados/singleton/{all/osd-backfill crimson-supported-all-distro/centos_latest crimson_qa_overrides objectstore/bluestore rados} time: 00:15:53 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420962/ [7420964] crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/fio_4K_rand_rw} time: 00:15:29 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420964/ [7420965] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/pool-snaps-few-objects} time: 00:27:55 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420965/ [7420966] crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/fio_4M_rand_read} time: 00:14:36 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420966/ [7420968] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/radosbench-high-concurrency} time: 00:25:07 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420968/ [7420969] crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/fio_4M_rand_rw} time: 00:16:01 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420969/ [7420970] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/radosbench} time: 00:25:57 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420970/ [7420971] crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/fio_4M_rand_write} time: 00:15:35 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420971/ [7420972] crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rados_python} time: 00:16:49 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420972/ [7420973] crimson-rados/rbd/{clusters/fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rbd_cls_tests} time: 00:12:14 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420973/ [7420974] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/small-objects-balanced} time: 00:20:19 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420974/ [7420976] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/small-objects-localized} time: 00:20:22 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420976/ [7420978] crimson-rados/rbd/{clusters/fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rbd_lock_and_fence} time: 00:11:12 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420978/ [7420980] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/small-objects} time: 00:20:12 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420980/ [7420982] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/snaps-few-objects-balanced} time: 00:28:29 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420982/ [7420983] crimson-rados/basic/{clusters/fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/readwrite} time: 00:14:02 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420983/ [7420984] crimson-rados/rbd/{clusters/fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rbd_python_api_tests} time: 00:22:17 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420984/ [7420988] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/snaps-few-objects} time: 00:29:11 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420988/ [7420989] crimson-rados/rbd/{clusters/fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph tasks/rbd_python_api_tests_old_format} time: 00:20:11 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420989/ [7420990] crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/sample_fio} time: 00:16:13 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420990/ [7420991] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/write_fadvise_dontneed} time: 00:14:57 info: http://pulpito.ceph.com/yingxin-2023-10-11_02:13:06-crimson-rados-wip-yingxin-crimson-improve-mempool-distro-default-smithi/7420991/