2024-01-09T22:51:48.818 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-01-09T22:51:48.818 INFO:teuthology.results:subset: '111/12000' 2024-01-09T22:51:48.818 INFO:teuthology.results:seed: '6252' 2024-01-10T10:10:22.301 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-01-10T10:10:22.331 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2024-01-10T12:54:38.440 INFO:teuthology.results:Tests finished! gathering results... 2024-01-10T12:55:10.322 INFO:scrape:Found 412 jobs 2024-01-10T12:55:16.135 INFO:scrape:Found 9 distinct failure reasons 2024-01-10T12:55:16.136 INFO:scrape:Failure: Command failed on smithi081 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone --depth 1 --branch pacific https://github.com/cbodley/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0' 2024-01-10T12:55:16.136 INFO:scrape:2 jobs: ['7510546', '7510149'] 2024-01-10T12:55:16.136 INFO:scrape:suites intersection: ['0-start', '1-tasks', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'test_rbd_api', 'test_rbd_python}}', 'upgrade-sequence', 'workload/{ec-rados-default'] 2024-01-10T12:55:16.136 INFO:scrape:suites union: ['0-start', '1-tasks', 'mon_election/classic', 'mon_election/connectivity', 'rados/upgrade/parallel/{0-random-distro$/{centos_8.stream_container_tools}', 'rados/upgrade/parallel/{0-random-distro$/{ubuntu_20.04}', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'test_rbd_api', 'test_rbd_python}}', 'upgrade-sequence', 'workload/{ec-rados-default'] 2024-01-10T12:55:16.137 INFO:scrape: 2024-01-10T12:55:16.137 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi053 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=affad57d5ae9c6b2f7579f7a454052bf9428828f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2024-01-10T12:55:16.137 INFO:scrape:2 jobs: ['7509979', '7510348'] 2024-01-10T12:55:16.137 INFO:scrape:suites intersection: ['debug/mgr', 'rados/dashboard/{0-single-container-host', 'tasks/e2e}'] 2024-01-10T12:55:16.137 INFO:scrape:suites union: ['debug/mgr', 'mon_election/classic', 'mon_election/connectivity', 'rados/dashboard/{0-single-container-host', 'random-objectstore$/{bluestore-low-osd-mem-target}', 'random-objectstore$/{bluestore-stupid}', 'tasks/e2e}'] 2024-01-10T12:55:16.137 INFO:scrape: 2024-01-10T12:55:16.137 INFO:scrape:Failure: ['Failed to update apt cache: unknown reason'] 2024-01-10T12:55:16.137 INFO:scrape:1 jobs: ['7510241'] 2024-01-10T12:55:16.137 INFO:scrape:suites: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{more-async-partial-recovery}', '3-scrub-overrides/{max-simultaneous-scrubs-2}', 'backoff/normal', 'ceph', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'd-balancer/crush-compat', 'mon_election/classic', 'msgr-failures/osd-dispatch-delay', 'msgr/async-v2only', 'objectstore/bluestore-bitmap', 'openstack}', 'rados', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/none', 'thrashosds-health', 'workloads/admin_socket_objecter_requests}'] 2024-01-10T12:55:16.137 INFO:scrape: 2024-01-10T12:55:16.138 INFO:scrape:Failure: Command failed (workunit test mon/health-mute.sh) on smithi072 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=affad57d5ae9c6b2f7579f7a454052bf9428828f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/mon/health-mute.sh' 2024-01-10T12:55:16.138 INFO:scrape:1 jobs: ['7510629'] 2024-01-10T12:55:16.138 INFO:scrape:suites: ['rados/standalone/{supported-random-distro$/{ubuntu_latest}', 'workloads/mon}'] 2024-01-10T12:55:16.138 INFO:scrape: 2024-01-10T12:55:16.138 INFO:scrape:Failure: Command failed (workunit test post-file.sh) on smithi181 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=affad57d5ae9c6b2f7579f7a454052bf9428828f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/post-file.sh' 2024-01-10T12:55:16.138 INFO:scrape:2 jobs: ['7510239', '7510641'] 2024-01-10T12:55:16.138 INFO:scrape:suites intersection: ['rados', 'rados/singleton-nomsgr/{all/ceph-post-file', 'supported-random-distro$/{ubuntu_latest}}'] 2024-01-10T12:55:16.138 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados', 'rados/singleton-nomsgr/{all/ceph-post-file', 'supported-random-distro$/{ubuntu_latest}}'] 2024-01-10T12:55:16.138 INFO:scrape: 2024-01-10T12:55:16.138 INFO:scrape:Failure: "1704865328.5946474 mon.a (mon.0) 70 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log 2024-01-10T12:55:16.139 INFO:scrape:1 jobs: ['7510371'] 2024-01-10T12:55:16.139 INFO:scrape:suites: ['rados/objectstore/{backends/filestore-idempotent-aio-journal', 'supported-random-distro$/{centos_8}}'] 2024-01-10T12:55:16.139 INFO:scrape: 2024-01-10T12:55:16.139 INFO:scrape:Failure: Command failed on smithi121 with status 1: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c \'mkdir $TESTDIR/archive/ostest && cd $TESTDIR/archive/ostest && ulimit -Sn 16384 && CEPH_ARGS="--no-log-to-stderr --log-file $TESTDIR/archive/ceph_test_objectstore.log --debug-bluestore 20" ceph_test_objectstore --gtest_filter=*/2:-*SyntheticMatrixC* --gtest_catch_exceptions=0\'' 2024-01-10T12:55:16.139 INFO:scrape:1 jobs: ['7509971'] 2024-01-10T12:55:16.139 INFO:scrape:suites: ['rados/objectstore/{backends/objectstore-bluestore-a', 'supported-random-distro$/{centos_8}}'] 2024-01-10T12:55:16.139 INFO:scrape: 2024-01-10T12:55:16.139 INFO:scrape:Failure: Command failed (workunit test misc/test-ceph-helpers.sh) on smithi057 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=affad57d5ae9c6b2f7579f7a454052bf9428828f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-ceph-helpers.sh' 2024-01-10T12:55:16.139 INFO:scrape:1 jobs: ['7510458'] 2024-01-10T12:55:16.139 INFO:scrape:suites: ['rados/standalone/{supported-random-distro$/{centos_8}', 'workloads/misc}'] 2024-01-10T12:55:16.139 INFO:scrape: 2024-01-10T12:55:16.140 INFO:scrape:Failure: Command failed (workunit test rados/test.sh) on smithi077 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=affad57d5ae9c6b2f7579f7a454052bf9428828f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' 2024-01-10T12:55:16.140 INFO:scrape:1 jobs: ['7510390'] 2024-01-10T12:55:16.140 INFO:scrape:suites: ['clusters/3-mons', 'mon_election/classic', 'msgr-failures/mon-delay', 'msgr/async-v2only', 'objectstore/filestore-xfs', 'rados', 'rados/monthrash/{ceph', 'supported-random-distro$/{rhel_8}', 'thrashers/one', 'workloads/rados_api_tests}'] 2024-01-10T12:55:16.140 INFO:scrape: 2024-01-10T12:55:16.154 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi ================================================================= info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/ failed: 11 dead: 1 running: 0 waiting: 0 queued: 0 passed: 400 Fail ================================================================= [7509971] rados/objectstore/{backends/objectstore-bluestore-a supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 02:48:59 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509971/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509971/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4e0ce10110194e7981ca4c591875ed5b Command failed on smithi121 with status 1: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c \'mkdir $TESTDIR/archive/ostest && cd $TESTDIR/archive/ostest && ulimit -Sn 16384 && CEPH_ARGS="--no-log-to- stderr --log-file $TESTDIR/archive/ceph_test_objectstore.log --debug- bluestore 20" ceph_test_objectstore --gtest_filter=*/2:-*SyntheticMatrixC* --gtest_catch_exceptions=0\'' [7509979] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-stupid} tasks/e2e} ----------------------------------------------------------------- time: 00:44:38 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509979/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509979/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f6785f3ee79644f2bc8bf8b4e073952e Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi053 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=affad57d5ae9c6b2f7579f7a454052bf9428828f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [7510149] rados/upgrade/parallel/{0-random-distro$/{ubuntu_20.04} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} ----------------------------------------------------------------- time: 00:36:08 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510149/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510149/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b09b2f42833b4c029a9075bb2e16aa1a Command failed on smithi153 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone --depth 1 --branch pacific https://github.com/cbodley/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0' [7510239] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:08:09 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510239/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510239/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d9ee33395eb140d0970ddea4ccdd864f Command failed (workunit test post-file.sh) on smithi181 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=affad57d5ae9c6b2f7579f7a454052bf9428828f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/post-file.sh' [7510348] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-low-osd-mem-target} tasks/e2e} ----------------------------------------------------------------- time: 00:18:39 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510348/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510348/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d406fad5e09f429687435624cd4301d2 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi099 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=affad57d5ae9c6b2f7579f7a454052bf9428828f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [7510371] rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 02:33:28 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510371/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510371/ "1704865328.5946474 mon.a (mon.0) 70 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log [7510390] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/one workloads/rados_api_tests} ----------------------------------------------------------------- time: 00:39:56 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510390/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510390/ sentry: https://sentry.ceph.com/organizations/ceph/?query=09b1a1116aa043bf8da11831e813436c Command failed (workunit test rados/test.sh) on smithi077 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=affad57d5ae9c6b2f7579f7a454052bf9428828f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [7510458] rados/standalone/{supported-random-distro$/{centos_8} workloads/misc} ----------------------------------------------------------------- time: 00:59:17 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510458/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510458/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ff353c5bf5234cec8770b87db091caa8 Command failed (workunit test misc/test-ceph-helpers.sh) on smithi057 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=affad57d5ae9c6b2f7579f7a454052bf9428828f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-ceph- helpers.sh' [7510546] rados/upgrade/parallel/{0-random-distro$/{centos_8.stream_container_tools} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} ----------------------------------------------------------------- time: 00:52:03 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510546/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510546/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cd8d7b3237654ec692a46160a90c3010 Command failed on smithi081 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone --depth 1 --branch pacific https://github.com/cbodley/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0' [7510629] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/mon} ----------------------------------------------------------------- time: 00:10:31 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510629/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510629/ sentry: https://sentry.ceph.com/organizations/ceph/?query=15b6975acafe48b1b3a55831666464e7 Command failed (workunit test mon/health-mute.sh) on smithi072 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=affad57d5ae9c6b2f7579f7a454052bf9428828f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/mon/health-mute.sh' [7510641] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:08:31 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510641/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510641/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4e334452eba64f169f39291b36eb8bba Command failed (workunit test post-file.sh) on smithi106 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=affad57d5ae9c6b2f7579f7a454052bf9428828f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/post-file.sh' Dead ================================================================= [7510241] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/admin_socket_objecter_requests} ----------------------------------------------------------------- time: 00:17:25 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510241/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510241/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f2a4737105cb42f791c529bd6197361a ['Failed to update apt cache: unknown reason'] Pass ================================================================= [7509964] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/write_fadvise_dontneed} time: 00:15:15 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509964/ [7509965] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-hybrid} supported-random-distro$/{ubuntu_latest} tasks/progress} time: 00:20:15 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509965/ [7509966] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/many workloads/rados_api_tests} time: 00:56:01 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509966/ [7509967] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:19:39 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509967/ [7509968] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:21:28 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509968/ [7509969] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/admin_socket_objecter_requests} time: 00:46:49 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509969/ [7509970] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-balanced} time: 00:49:15 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509970/ [7509972] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:26:20 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509972/ [7509973] rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:19:22 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509973/ [7509974] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_mix} time: 00:24:11 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509974/ [7509975] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-agent-big} time: 00:25:00 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509975/ [7509976] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache-agent-small} time: 00:49:44 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509976/ [7509977] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:15:30 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509977/ [7509978] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:50:13 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509978/ [7509980] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:17:34 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509980/ [7509981] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-pool-snaps} time: 00:50:19 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509981/ [7509982] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:19:07 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509982/ [7509983] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache-snaps-balanced} time: 00:49:23 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509983/ [7509984] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_orch_cli_mon} time: 00:50:25 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509984/ [7509985] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:12:17 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509985/ [7509986] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:28:13 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509986/ [7509987] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:18:36 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509987/ [7509988] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache} time: 00:37:42 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509988/ [7509989] rados/singleton/{all/ec-inconsistent-hinfo mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:16:50 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509989/ [7509990] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/dedup-io-mixed} time: 00:11:45 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509990/ [7509991] rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0} 1-start 2-services/rgw 3-final} time: 00:17:00 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509991/ [7509992] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/dedup-io-snaps} time: 00:15:34 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509992/ [7509993] rados/standalone/{supported-random-distro$/{rhel_8} workloads/osd} time: 02:49:44 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509993/ [7509994] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/pool-snaps-few-objects} time: 00:59:15 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509994/ [7509995] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:21:09 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509995/ [7509996] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:33:19 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509996/ [7509997] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mostlyread} time: 00:29:24 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509997/ [7509998] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:19:00 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509998/ [7509999] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/rados_api_tests} time: 00:56:13 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7509999/ [7510000] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{rhel_8} tasks/prometheus} time: 00:20:26 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510000/ [7510001] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:36:07 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510001/ [7510002] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:32:32 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510002/ [7510003] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:19:11 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510003/ [7510004] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} time: 01:23:20 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510004/ [7510005] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/radosbench-high-concurrency} time: 00:30:48 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510005/ [7510006] rados/objectstore/{backends/objectstore-bluestore-b supported-random-distro$/{ubuntu_latest}} time: 02:50:57 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510006/ [7510007] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/rados_mon_osdmap_prune} time: 00:55:08 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510007/ [7510008] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:42:16 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510008/ [7510009] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-small-objects-fast-read} time: 00:49:50 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510009/ [7510010] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/radosbench} time: 00:34:29 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510010/ [7510011] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} time: 00:14:59 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510011/ [7510012] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:17:03 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510012/ [7510013] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:24:16 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510013/ [7510015] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/redirect} time: 00:18:36 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510015/ [7510017] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/snaps-few-objects} time: 01:04:38 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510017/ [7510019] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:11:39 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510019/ [7510021] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/redirect_promote_tests} time: 00:12:30 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510021/ [7510023] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:51:24 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510023/ [7510025] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/redirect_set_object} time: 00:46:48 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510025/ [7510027] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_rgw_multisite} time: 00:38:44 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510027/ [7510029] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/set-chunks-read} time: 00:44:14 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510029/ [7510031] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:52:59 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510031/ [7510033] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:28:44 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510033/ [7510035] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/small-objects-balanced} time: 00:29:05 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510035/ [7510037] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:52:05 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510037/ [7510039] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:16:32 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510039/ [7510041] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/small-objects-localized} time: 00:28:40 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510041/ [7510043] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/small-objects} time: 00:30:12 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510043/ [7510045] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:29:09 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510045/ [7510047] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} tasks/readwrite} time: 00:26:44 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510047/ [7510049] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/snaps-few-objects-balanced} time: 00:36:06 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510049/ [7510051] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:30:57 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510051/ [7510053] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_set_mon_crush_locations} time: 00:25:38 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510053/ [7510055] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/snaps-few-objects-localized} time: 00:26:20 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510055/ [7510057] rados/standalone/{supported-random-distro$/{centos_8} workloads/scrub} time: 02:30:48 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510057/ [7510059] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:34:06 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510059/ [7510061] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{rhel_8} tasks/workunits} time: 00:17:46 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510061/ [7510063] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:12:15 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510063/ [7510065] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:11:17 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510065/ [7510067] rados/objectstore/{backends/objectstore-filestore-memstore supported-random-distro$/{rhel_8}} time: 00:38:10 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510067/ [7510069] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/write_fadvise_dontneed} time: 00:14:19 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510069/ [7510071] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:17:49 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510071/ [7510073] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:21:48 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510073/ [7510075] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:35:16 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510075/ [7510077] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:17:12 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510077/ [7510079] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep} time: 00:32:38 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510079/ [7510081] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/rados_mon_workunits} time: 00:39:08 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510081/ [7510083] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:18:01 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510083/ [7510085] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:26:35 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510085/ [7510087] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/admin_socket_objecter_requests} time: 00:15:42 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510087/ [7510089] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:28:29 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510089/ [7510091] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-agent-big} time: 00:48:42 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510091/ [7510093] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_adoption} time: 00:17:36 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510093/ [7510095] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache-agent-small} time: 00:28:28 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510095/ [7510097] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest}} time: 00:12:09 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510097/ [7510099] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:23:20 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510099/ [7510101] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:31:08 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510101/ [7510103] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps} time: 00:38:39 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510103/ [7510105] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:23:23 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510105/ [7510107] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-snaps-balanced} time: 00:37:45 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510107/ [7510109] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:12:05 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510109/ [7510111] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} tasks/repair_test} time: 00:32:02 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510111/ [7510113] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-snaps} time: 00:26:22 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510113/ [7510115] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:12:57 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510115/ [7510117] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache} time: 00:20:14 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510117/ [7510119] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/test_rbd_api} time: 00:30:59 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510119/ [7510121] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_cephadm} time: 00:21:08 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510121/ [7510123] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:12:04 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510123/ [7510125] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/dedup-io-mixed} time: 00:29:13 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510125/ [7510127] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:11:53 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510127/ [7510129] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:31:25 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510129/ [7510131] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/dedup-io-snaps} time: 00:28:03 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510131/ [7510133] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:16:21 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510133/ [7510135] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/pool-snaps-few-objects} time: 00:25:25 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510135/ [7510137] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-zlib} tasks/dashboard} time: 01:02:22 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510137/ [7510139] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:11:11 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510139/ [7510141] rados/objectstore/{backends/alloc-hint supported-random-distro$/{ubuntu_latest}} time: 00:10:38 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510141/ [7510143] rados/rest/{mgr-restful supported-random-distro$/{ubuntu_latest}} time: 00:16:20 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510143/ [7510145] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:23:35 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510145/ [7510147] rados/standalone/{supported-random-distro$/{rhel_8} workloads/c2c} time: 00:17:25 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510147/ [7510151] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:26:12 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510151/ [7510153] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/rados_api_tests} time: 00:33:59 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510153/ [7510154] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:10:50 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510154/ [7510156] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/radosbench-high-concurrency} time: 00:42:04 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510156/ [7510158] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:36:50 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510158/ [7510160] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:38:17 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510160/ [7510162] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:18:20 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510162/ [7510164] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} time: 00:34:12 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510164/ [7510166] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/sync workloads/snaps-few-objects} time: 00:33:35 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510166/ [7510168] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:22 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510168/ [7510170] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects} time: 00:20:58 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510170/ [7510172] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:14:31 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510172/ [7510174] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/radosbench} time: 00:32:45 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510174/ [7510176] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/redirect} time: 00:17:43 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510176/ [7510178] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:17:37 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510178/ [7510180] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/redirect_promote_tests} time: 00:21:56 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510180/ [7510182] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:22:03 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510182/ [7510185] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/scrub_test} time: 00:11:58 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510185/ [7510186] rados/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_20.04} 1-start 2-services/basic 3-final} time: 00:16:59 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510186/ [7510188] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/redirect_set_object} time: 00:12:48 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510188/ [7510190] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:12:31 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510190/ [7510192] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/set-chunks-read} time: 00:23:26 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510192/ [7510194] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:16:25 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510194/ [7510197] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_cephadm_repos} time: 00:09:07 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510197/ [7510199] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/small-objects-balanced} time: 00:27:10 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510199/ [7510201] rados/singleton/{all/mon-config mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:15:46 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510201/ [7510203] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/small-objects-localized} time: 00:20:08 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510203/ [7510205] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/small-objects} time: 00:28:41 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510205/ [7510207] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:13:31 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510207/ [7510209] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/snaps-few-objects-balanced} time: 00:35:37 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510209/ [7510211] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:17:50 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510211/ [7510213] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:22:35 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510213/ [7510215] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/snaps-few-objects-localized} time: 00:24:59 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510215/ [7510217] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{centos_8}} time: 00:17:34 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510217/ [7510219] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{ubuntu_latest} tasks/failover} time: 00:12:44 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510219/ [7510221] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} time: 00:27:53 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510221/ [7510223] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:15:36 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510223/ [7510225] rados/standalone/{supported-random-distro$/{centos_8} workloads/crush} time: 00:24:17 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510225/ [7510227] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/write_fadvise_dontneed} time: 00:15:26 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510227/ [7510229] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/libcephsqlite} time: 00:17:20 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510229/ [7510231] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/pool-create-delete} time: 00:17:21 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510231/ [7510233] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:23:27 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510233/ [7510235] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:26:05 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510235/ [7510237] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:34:26 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510237/ [7510243] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:30:32 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510243/ [7510245] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:13:55 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510245/ [7510247] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:54 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510247/ [7510249] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:20:44 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510249/ [7510251] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/cache-snaps} time: 00:29:45 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510251/ [7510253] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} time: 00:26:28 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510253/ [7510255] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:12:47 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510255/ [7510257] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:13:38 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510257/ [7510259] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache-agent-big} time: 00:27:02 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510259/ [7510261] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache-agent-small} time: 00:14:17 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510261/ [7510263] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:29:19 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510263/ [7510265] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:21:37 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510265/ [7510267] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:28:37 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510267/ [7510269] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_orch_cli} time: 00:23:04 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510269/ [7510271] rados/singleton-nomsgr/{all/crushdiff mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:11:26 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510271/ [7510273] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps} time: 00:26:26 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510273/ [7510275] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-snaps-balanced} time: 00:28:26 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510275/ [7510277] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache-snaps} time: 00:19:26 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510277/ [7510279] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:18:21 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510279/ [7510281] rados/singleton/{all/peer mon_election/classic msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:16:22 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510281/ [7510283] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache} time: 00:21:46 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510283/ [7510285] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:10:27 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510285/ [7510287] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/dedup-io-mixed} time: 00:20:15 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510287/ [7510289] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:18:48 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510289/ [7510291] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/dedup-io-snaps} time: 00:14:49 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510291/ [7510292] rados/objectstore/{backends/filejournal supported-random-distro$/{centos_8}} time: 00:10:34 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510292/ [7510294] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:10:38 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510294/ [7510297] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/rados_api_tests} time: 00:27:31 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510297/ [7510299] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:16:54 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510299/ [7510300] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-bitmap} supported-random-distro$/{rhel_8} tasks/insights} time: 00:20:16 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510300/ [7510302] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/pool-snaps-few-objects} time: 00:36:38 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510302/ [7510304] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:11:23 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510304/ [7510306] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/rados_api_tests} time: 00:35:48 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510306/ [7510308] rados/standalone/{supported-random-distro$/{centos_8} workloads/erasure-code} time: 01:28:13 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510308/ [7510310] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_orch_cli_mon} time: 00:31:00 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510310/ [7510312] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/radosbench-high-concurrency} time: 00:25:48 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510312/ [7510314] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:24:04 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510314/ [7510316] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:21:33 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510316/ [7510318] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/many workloads/rados_5925} time: 00:19:19 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510318/ [7510320] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:10:27 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510320/ [7510322] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} time: 02:17:25 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510322/ [7510324] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:27:48 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510324/ [7510326] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:11 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510326/ [7510328] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/radosbench} time: 00:49:39 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510328/ [7510330] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:16:17 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510330/ [7510332] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/redirect} time: 00:14:07 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510332/ [7510334] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:18:45 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510334/ [7510336] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:10:08 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510336/ [7510338] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/redirect_promote_tests} time: 00:20:18 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510338/ [7510340] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/redirect_set_object} time: 00:16:47 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510340/ [7510342] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:09:00 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510342/ [7510344] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:16:04 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510344/ [7510346] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/set-chunks-read} time: 00:17:27 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510346/ [7510350] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:16:55 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510350/ [7510352] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/small-objects-balanced} time: 00:19:44 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510352/ [7510354] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v1only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/radosbench} time: 01:28:30 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510354/ [7510356] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:21:14 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510356/ [7510358] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/small-objects-localized} time: 00:20:50 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510358/ [7510360] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_rgw_multisite} time: 00:18:03 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510360/ [7510362] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/rados_cls_all} time: 00:18:03 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510362/ [7510364] rados/singleton/{all/radostool mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:15:14 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510364/ [7510367] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/small-objects} time: 00:20:37 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510367/ [7510369] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites} time: 00:22:28 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510369/ [7510372] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/snaps-few-objects-balanced} time: 00:34:29 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510372/ [7510374] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:10:12 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510374/ [7510376] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects-localized} time: 00:28:33 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510376/ [7510378] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{rhel_8} tasks/module_selftest} time: 00:33:06 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510378/ [7510380] rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0} 1-start 2-services/rgw 3-final} time: 00:17:08 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510380/ [7510381] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:30:30 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510381/ [7510382] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} time: 00:25:43 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510382/ [7510383] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/write_fadvise_dontneed} time: 00:13:43 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510383/ [7510384] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/mgr} time: 00:15:37 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510384/ [7510385] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_set_mon_crush_locations} time: 00:17:30 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510385/ [7510386] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/admin_socket_objecter_requests} time: 00:18:54 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510386/ [7510387] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:23:18 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510387/ [7510388] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:23 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510388/ [7510389] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:21:52 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510389/ [7510391] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:12:32 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510391/ [7510392] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} time: 00:15:48 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510392/ [7510393] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:11:47 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510393/ [7510394] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:24:00 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510394/ [7510396] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-agent-big} time: 00:16:27 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510396/ [7510398] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:20:24 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510398/ [7510400] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-agent-small} time: 00:15:37 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510400/ [7510402] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:12:41 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510402/ [7510404] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:15:31 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510404/ [7510406] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:34:31 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510406/ [7510408] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:22:56 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510408/ [7510410] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:23:03 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510410/ [7510412] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-pool-snaps} time: 00:22:10 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510412/ [7510414] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} tasks/rados_python} time: 00:19:02 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510414/ [7510416] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:18:05 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510416/ [7510418] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-snaps-balanced} time: 00:27:52 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510418/ [7510420] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:23:42 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510420/ [7510422] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache-snaps} time: 00:22:45 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510422/ [7510424] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:32:27 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510424/ [7510426] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/cache} time: 00:23:07 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510426/ [7510428] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:30:04 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510428/ [7510430] rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{ubuntu_latest}} time: 02:18:05 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510430/ [7510432] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:10:21 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510432/ [7510434] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_adoption} time: 00:11:27 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510434/ [7510436] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/dedup-io-mixed} time: 00:21:01 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510436/ [7510438] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/dedup-io-snaps} time: 00:20:53 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510438/ [7510440] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-stupid} supported-random-distro$/{rhel_8} tasks/per_module_finisher_stats} time: 00:17:54 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510440/ [7510442] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/pool-snaps-few-objects} time: 00:26:58 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510442/ [7510444] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:17:49 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510444/ [7510446] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:26:58 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510446/ [7510448] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/rados_api_tests} time: 00:41:00 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510448/ [7510450] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/rbd_cls} time: 00:18:19 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510450/ [7510452] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:18:21 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510452/ [7510454] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:10:54 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510454/ [7510456] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/radosbench-high-concurrency} time: 00:46:55 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510456/ [7510460] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-radosbench} time: 01:05:34 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510460/ [7510462] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:21:30 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510462/ [7510464] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/sync-many workloads/rados_mon_osdmap_prune} time: 00:47:12 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510464/ [7510466] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/radosbench} time: 00:53:39 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510466/ [7510468] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:13:26 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510468/ [7510470] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:17:44 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510470/ [7510472] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} time: 01:03:59 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510472/ [7510474] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:32:33 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510474/ [7510476] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:32:00 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510476/ [7510478] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:35:10 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510478/ [7510480] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/redirect} time: 00:23:33 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510480/ [7510482] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/rados_stress_watch} time: 00:18:56 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510482/ [7510484] rados/singleton/{all/test-noautoscale-flag mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:09:25 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510484/ [7510486] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:13:42 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510486/ [7510488] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/redirect_promote_tests} time: 00:24:58 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510488/ [7510490] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_cephadm} time: 00:26:38 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510490/ [7510492] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/redirect_set_object} time: 00:16:12 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510492/ [7510494] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/set-chunks-read} time: 00:17:37 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510494/ [7510496] rados/singleton/{all/test_envlibrados_for_rocksdb mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:27:14 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510496/ [7510498] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/small-objects-balanced} time: 00:25:59 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510498/ [7510500] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:33:35 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510500/ [7510502] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_cephadm_repos} time: 00:12:19 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510502/ [7510504] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/small-objects-localized} time: 00:28:57 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510504/ [7510506] rados/objectstore/{backends/fusestore supported-random-distro$/{centos_8}} time: 00:17:23 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510506/ [7510508] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/small-objects} time: 00:30:37 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510508/ [7510510] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:13:06 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510510/ [7510512] rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 01:23:22 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510512/ [7510514] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/snaps-few-objects-balanced} time: 00:26:42 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510514/ [7510516] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:16:33 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510516/ [7510518] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{filestore-xfs} supported-random-distro$/{rhel_8} tasks/progress} time: 00:26:59 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510518/ [7510520] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:19:39 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510520/ [7510522] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-localized} time: 00:40:16 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510522/ [7510524] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/snaps-few-objects} time: 00:24:38 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510524/ [7510526] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:19:07 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510526/ [7510528] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/write_fadvise_dontneed} time: 00:29:43 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510528/ [7510530] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} tasks/rados_striper} time: 00:13:03 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510530/ [7510532] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:37:03 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510532/ [7510534] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/admin_socket_objecter_requests} time: 00:37:00 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510534/ [7510536] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-small-objects-balanced} time: 00:29:21 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510536/ [7510538] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:39:35 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510538/ [7510540] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-bitmap} tasks/dashboard} time: 01:31:36 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510540/ [7510542] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:43:20 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510542/ [7510544] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/mon-stretch} time: 00:21:18 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510544/ [7510548] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/sync workloads/rados_mon_workunits} time: 00:51:40 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510548/ [7510550] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-agent-big} time: 00:19:25 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510550/ [7510552] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:39:04 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510552/ [7510554] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/rados_cls_all validater/lockdep} time: 00:46:23 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510554/ [7510556] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:19:10 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510556/ [7510558] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:33:41 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510558/ [7510560] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:31:23 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510560/ [7510562] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache-agent-small} time: 00:33:37 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510562/ [7510564] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:38:35 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510564/ [7510566] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:35:59 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510566/ [7510568] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:10:27 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510568/ [7510570] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-pool-snaps} time: 00:28:21 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510570/ [7510572] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/snaps-few-objects} time: 00:34:49 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510572/ [7510574] rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0} 1-start 2-services/basic 3-final} time: 00:15:52 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510574/ [7510576] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:12:54 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510576/ [7510578] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache-snaps-balanced} time: 00:28:57 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510578/ [7510580] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:22:17 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510580/ [7510582] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/cache-snaps} time: 00:33:31 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510582/ [7510584] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{centos_8}} time: 00:15:53 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510584/ [7510586] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:41:57 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510586/ [7510588] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:12:55 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510588/ [7510590] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache} time: 00:11:43 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510590/ [7510592] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:16:11 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510592/ [7510594] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/dedup-io-mixed} time: 00:14:15 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510594/ [7510596] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{ubuntu_latest} tasks/prometheus} time: 00:11:10 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510596/ [7510598] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/dedup-io-snaps} time: 00:21:25 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510598/ [7510600] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_big} time: 00:31:33 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510600/ [7510602] rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:17:08 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510602/ [7510604] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:18:48 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510604/ [7510606] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/pool-snaps-few-objects} time: 00:26:58 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510606/ [7510608] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/rados_api_tests} time: 00:27:55 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510608/ [7510610] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:16:50 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510610/ [7510612] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/radosbench-high-concurrency} time: 00:26:12 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510612/ [7510614] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_orch_cli} time: 00:14:59 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510614/ [7510616] rados/singleton/{all/admin-socket mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:10:57 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510616/ [7510618] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:32:25 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510618/ [7510620] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read} time: 00:22:11 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510620/ [7510622] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:09 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510622/ [7510624] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/radosbench} time: 00:45:25 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510624/ [7510626] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/snaps-few-objects} time: 00:29:24 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510626/ [7510628] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:11:28 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510628/ [7510631] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:16:50 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510631/ [7510633] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados tasks/mon_recovery validater/valgrind} time: 00:35:09 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510633/ [7510635] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/redirect} time: 00:22:40 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510635/ [7510637] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:25:19 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510637/ [7510639] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:47 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510639/ [7510643] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_orch_cli_mon} time: 00:23:59 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510643/ [7510645] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/redirect_promote_tests} time: 00:15:08 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510645/ [7510647] rados/singleton/{all/backfill-toofull mon_election/classic msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:26:26 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510647/ [7510649] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/redirect_set_object} time: 00:20:36 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510649/ [7510651] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/set-chunks-read} time: 00:15:04 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510651/ [7510653] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/repave-all} time: 00:12:32 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510653/ [7510655] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/small-objects-balanced} time: 00:24:28 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510655/ [7510657] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{rhel_8}} time: 00:33:01 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510657/ [7510659] rados/singleton-nomsgr/{all/crushdiff mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:09:22 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510659/ [7510661] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mix} time: 00:21:33 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510661/ [7510663] rados/singleton/{all/deduptool mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:09:54 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510663/ [7510665] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/small-objects-localized} time: 00:24:17 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510665/ [7510667] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/small-objects} time: 00:21:04 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510667/ [7510669] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:16:48 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510669/ [7510671] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-hybrid} supported-random-distro$/{ubuntu_latest} tasks/workunits} time: 00:10:06 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510671/ [7510673] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-balanced} time: 00:33:40 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510673/ [7510675] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:08:44 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510675/ [7510677] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/snaps-few-objects-localized} time: 00:24:21 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510677/ [7510679] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/octopus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/test_rbd_api} time: 00:19:01 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510679/ [7510681] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:22:56 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510681/ [7510683] rados/singleton/{all/divergent_priors mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:19:12 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510683/ [7510685] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:11:43 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510685/ [7510687] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_rgw_multisite} time: 00:25:29 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510687/ [7510689] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/snaps-few-objects} time: 00:35:26 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510689/ [7510691] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/write_fadvise_dontneed} time: 00:24:51 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510691/ [7510693] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:30:21 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510693/ [7510695] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/admin_socket_objecter_requests} time: 00:19:56 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510695/ [7510697] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:20:53 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510697/ [7510699] rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:10:28 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510699/ [7510701] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:24:49 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510701/ [7510703] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:21:55 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510703/ [7510705] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:09:08 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510705/ [7510707] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/many workloads/pool-create-delete} time: 00:21:29 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510707/ [7510709] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-agent-big} time: 01:00:45 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510709/ [7510711] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd-backfill} time: 02:39:34 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510711/ [7510713] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:21:39 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510713/ [7510715] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados tasks/mon_recovery validater/lockdep} time: 00:13:25 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510715/ [7510717] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-agent-small} time: 00:14:12 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510717/ [7510719] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:11:19 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510719/ [7510721] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:21 info: https://pulpito.ceph.com/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510721/