2023-06-24T04:22:14.155 INFO:root:teuthology version: 0.0.1.dev128+g076bbeb 2023-06-24T04:22:14.156 INFO:teuthology.results:subset: '111/120000' 2023-06-24T04:22:14.156 INFO:teuthology.results:seed: '9416' 2023-06-24T14:36:19.650 INFO:root:teuthology version: 0.0.1.dev128+g076bbeb 2023-06-24T14:36:19.664 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-06-24T17:19:45.827 INFO:teuthology.results:Tests finished! gathering results... 2023-06-24T17:20:08.349 INFO:scrape:Found 307 jobs 2023-06-24T17:20:09.784 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315283/teuthology.log 2023-06-24T17:20:10.157 INFO:scrape:Found 7 distinct failure reasons 2023-06-24T17:20:10.158 INFO:scrape:Failure: No module named 'tasks.ceph' 2023-06-24T17:20:10.158 INFO:scrape:4 jobs: ['7315090', '7315095', '7315109', '7315079'] 2023-06-24T17:20:10.158 INFO:scrape:suites intersection: [] 2023-06-24T17:20:10.159 INFO:scrape:suites union: ['1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{more-active-recovery}', '3-scrub-overrides/{max-simultaneous-scrubs-2}', 'backoff/peering', 'ceph', 'cluster/{12-osds', 'clusters/{fixed-2', 'crc-failures/default', 'd-balancer/on', 'mon_election/connectivity', 'msgr-failures/many', 'msgr-failures/osd-delay', 'msgr-failures/osd-dispatch-delay', 'msgr/async', 'msgr/async-v2only', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/singleton/{all/test-crash', 'rados/standalone/{supported-random-distro$/{rhel_8}', 'rados/thrash-erasure-code-big/{ceph', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'recovery-overrides/{more-async-recovery}', 'supported-random-distro$/{centos_8}}', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{ubuntu_20.04}', 'thrashers/default', 'thrashers/morepggrow', 'thrashosds-health', 'workloads/cache-pool-snaps-readproxy}', 'workloads/ec-rados-plugin=jerasure-k=4-m=2}', 'workloads/mon-stretch}'] 2023-06-24T17:20:10.159 INFO:scrape: 2023-06-24T17:20:10.159 INFO:scrape:Failure: Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_lazy_omap_stats' 2023-06-24T17:20:10.160 INFO:scrape:1 jobs: ['7315281'] 2023-06-24T17:20:10.160 INFO:scrape:suites: ['mon_election/connectivity', 'rados', 'rados/singleton-nomsgr/{all/lazy_omap_stats_output', 'supported-random-distro$/{ubuntu_latest}}'] 2023-06-24T17:20:10.160 INFO:scrape: 2023-06-24T17:20:10.161 INFO:scrape:Failure: Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi150 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b9d28de6006117997208584eb86d8d304f2a113d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh' 2023-06-24T17:20:10.161 INFO:scrape:2 jobs: ['7315215', '7315119'] 2023-06-24T17:20:10.161 INFO:scrape:suites intersection: ['mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v2only', 'rados', 'test_envlibrados_for_rocksdb}'] 2023-06-24T17:20:10.161 INFO:scrape:suites union: ['mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v2only', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-low-osd-mem-target', 'rados', 'rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/centos_latest', 'rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/rhel_latest', 'supported-random-distro$/{centos_8}}', 'supported-random-distro$/{rhel_8}}', 'test_envlibrados_for_rocksdb}'] 2023-06-24T17:20:10.162 INFO:scrape: 2023-06-24T17:20:10.162 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi002 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b9d28de6006117997208584eb86d8d304f2a113d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2023-06-24T17:20:10.162 INFO:scrape:2 jobs: ['7315261', '7314971'] 2023-06-24T17:20:10.163 INFO:scrape:suites intersection: ['debug/mgr', 'rados/dashboard/{0-single-container-host', 'random-objectstore$/{bluestore-comp-snappy}', 'tasks/e2e}'] 2023-06-24T17:20:10.163 INFO:scrape:suites union: ['debug/mgr', 'mon_election/classic', 'mon_election/connectivity', 'rados/dashboard/{0-single-container-host', 'random-objectstore$/{bluestore-comp-snappy}', 'tasks/e2e}'] 2023-06-24T17:20:10.163 INFO:scrape: 2023-06-24T17:20:10.164 INFO:scrape:Failure: reached maximum tries (121) after waiting for 120 seconds 2023-06-24T17:20:10.164 INFO:scrape:1 jobs: ['7314687'] 2023-06-24T17:20:10.164 INFO:scrape:suites: ['0-nvme-loop', 'agent/off', 'fixed-2', 'mon_election/connectivity', 'rados/cephadm/smoke/{0-distro/ubuntu_20.04', 'start}'] 2023-06-24T17:20:10.165 INFO:scrape: 2023-06-24T17:20:10.165 INFO:scrape:Dead: None 2023-06-24T17:20:10.165 INFO:scrape:2 jobs: ['7315283', '7315295'] 2023-06-24T17:20:10.166 INFO:scrape: 2023-06-24T17:20:10.166 INFO:scrape:Failure: Command failed on smithi052 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60cef6d6-1292-11ee-9b2b-001a4aab830c -- ceph mon dump -f json' 2023-06-24T17:20:10.166 INFO:scrape:1 jobs: ['7315194'] 2023-06-24T17:20:10.166 INFO:scrape:suites: ['0-nvme-loop', 'agent/off', 'fixed-2', 'mon_election/classic', 'rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8', 'start}'] 2023-06-24T17:20:10.167 INFO:scrape: 2023-06-24T17:20:10.180 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/ failed: 11 dead: 2 running: 0 waiting: 0 queued: 0 passed: 294 Fail ================================================================= [7314687] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:41:04 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314687/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314687/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ea45a7f885e640f285153ab9e8523046 reached maximum tries (121) after waiting for 120 seconds [7314971] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} tasks/e2e} ----------------------------------------------------------------- time: 00:26:43 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314971/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314971/ sentry: https://sentry.ceph.com/organizations/ceph/?query=48078f9bb3004f60b7d228aeb3a64f63 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi136 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b9d28de6006117997208584eb86d8d304f2a113d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [7315079] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} ----------------------------------------------------------------- time: 00:21:39 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315079/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315079/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bf40d76ba6a841c8b77bdfd4f0d8af60 No module named 'tasks.thrashosds' [7315090] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:14:50 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315090/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315090/ sentry: https://sentry.ceph.com/organizations/ceph/?query=61c645d168f0416e875c37eeee9c5892 No module named 'tasks.ceph' [7315095] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy} ----------------------------------------------------------------- time: 00:09:16 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315095/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315095/ sentry: https://sentry.ceph.com/organizations/ceph/?query=aebe0161f4354d80851fef2b7095bb01 No module named 'tasks.thrashosds' [7315109] rados/standalone/{supported-random-distro$/{rhel_8} workloads/mon-stretch} ----------------------------------------------------------------- time: 00:13:25 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315109/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315109/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e01cc6a6146441d3890a1a073081b693 No module named 'tasks' [7315119] rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/centos_latest test_envlibrados_for_rocksdb} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:13:22 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315119/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315119/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cdafaa27587a4dceb67b7cb280a656c0 Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi152 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b9d28de6006117997208584eb86d8d304f2a113d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh' [7315194] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:16:16 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315194/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315194/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e5bafe6f48da4d95a1a9c1e15721cdea Command failed on smithi052 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay- quay.apps.os.sepia.ceph.com/ceph- ci/ceph:c9d917e3a4a8fddd8a281c771f80a9b76e3a6fb8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 60cef6d6-1292-11ee-9b2b-001a4aab830c -- ceph mon dump -f json' [7315215] rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/rhel_latest test_envlibrados_for_rocksdb} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:14:05 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315215/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315215/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fcbe0f52555e489c976b1c5f39bb2f6e Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi150 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b9d28de6006117997208584eb86d8d304f2a113d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh' [7315261] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-snappy} tasks/e2e} ----------------------------------------------------------------- time: 00:28:35 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315261/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315261/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c013809c37444df08ab3845f20dcb1b9 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi002 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b9d28de6006117997208584eb86d8d304f2a113d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [7315281] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:11:33 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315281/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315281/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2f319d52b6fe4ec8814aa3ea4a0d348d Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_lazy_omap_stats' Dead ================================================================= [7315283] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315283/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315283/ Error reimaging machines: Failed to power on smithi033 [7315295] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315295/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315295/ Error reimaging machines: Failed to power on smithi171 Pass ================================================================= [7314592] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/rados_mon_osdmap_prune} time: 00:46:17 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314592/ [7314594] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/cache-snaps-balanced} time: 00:24:52 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314594/ [7314596] rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:12:23 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314596/ [7314598] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:13:14 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314598/ [7314600] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/snaps-few-objects} time: 00:34:54 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314600/ [7314602] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_cephadm_repos} time: 00:13:07 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314602/ [7314604] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:20:30 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314604/ [7314606] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:20:57 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314606/ [7314608] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:15:58 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314608/ [7314610] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} time: 00:16:23 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314610/ [7314612] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:21:40 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314612/ [7314614] rados/objectstore/{backends/objectstore-memstore supported-random-distro$/{ubuntu_latest}} time: 00:10:18 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314614/ [7314616] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:32:05 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314616/ [7314617] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/fastread thrashosds-health workloads/ec-small-objects} time: 00:21:14 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314617/ [7314619] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:18:19 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314619/ [7314621] rados/singleton/{all/ec-inconsistent-hinfo mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04}} time: 00:17:22 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314621/ [7314623] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:29:10 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314623/ [7314625] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache} time: 00:14:10 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314625/ [7314627] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:16:35 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314627/ [7314629] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite} time: 00:17:19 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314629/ [7314631] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:19:27 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314631/ [7314633] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:20:47 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314633/ [7314635] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados tasks/rados_cls_all validater/valgrind} time: 02:06:22 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314635/ [7314639] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:13:45 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314639/ [7314642] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:36:12 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314642/ [7314645] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:15:13 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314645/ [7314648] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/scrub} time: 02:50:48 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314648/ [7314651] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{rhel_8} tasks/workunits} time: 00:19:19 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314651/ [7314654] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:18:49 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314654/ [7314656] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/dedup-io-mixed} time: 00:17:23 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314656/ [7314659] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:27:03 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314659/ [7314662] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:11:10 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314662/ [7314665] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:13:08 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314665/ [7314668] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/many workloads/rados_mon_workunits} time: 00:35:54 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314668/ [7314672] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_nfs} time: 00:31:36 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314672/ [7314675] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/dedup-io-snaps} time: 00:13:55 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314675/ [7314678] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:10:48 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314678/ [7314681] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:32:14 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314681/ [7314684] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:30:38 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314684/ [7314690] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:14:02 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314690/ [7314693] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/rados_api_tests} time: 00:30:33 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314693/ [7314696] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/pool-snaps-few-objects} time: 00:33:02 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314696/ [7314699] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_orch_cli} time: 00:19:26 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314699/ [7314702] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/test_rbd_api} time: 00:21:36 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314702/ [7314705] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:31:19 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314705/ [7314708] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:12:35 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314708/ [7314711] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:25:02 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314711/ [7314714] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:34:48 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314714/ [7314717] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:47 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314717/ [7314720] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:15:22 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314720/ [7314723] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:37:08 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314723/ [7314726] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/rados_api_tests} time: 00:25:58 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314726/ [7314729] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:16:12 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314729/ [7314732] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-bitmap} tasks/dashboard} time: 01:06:44 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314732/ [7314735] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:13:29 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314735/ [7314738] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{centos_8}} time: 00:49:38 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314738/ [7314741] rados/rest/{mgr-restful supported-random-distro$/{rhel_8}} time: 00:18:11 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314741/ [7314744] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:22:36 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314744/ [7314747] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/c2c} time: 00:13:57 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314747/ [7314750] rados/upgrade/parallel/{0-random-distro$/{rhel_8.6_container_tools_3.0} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:32:54 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314750/ [7314753] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:21:36 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314753/ [7314756] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados tasks/mon_recovery validater/lockdep} time: 00:19:46 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314756/ [7314759] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_orch_cli_mon} time: 00:31:08 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314759/ [7314762] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:17:00 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314762/ [7314765] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:15:59 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314765/ [7314768] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/radosbench-high-concurrency} time: 00:35:36 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314768/ [7314771] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:10:37 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314771/ [7314774] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:16:09 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314774/ [7314778] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:14:35 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314778/ [7314781] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/one workloads/snaps-few-objects} time: 00:39:03 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314781/ [7314784] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} tasks/rados_cls_all} time: 00:25:41 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314784/ [7314786] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:16:00 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314786/ [7314789] rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_rhel8} 1-start 2-services/basic 3-final} time: 00:20:02 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314789/ [7314793] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:19:10 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314793/ [7314796] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/radosbench} time: 00:32:27 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314796/ [7314799] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:23:23 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314799/ [7314802] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:14:21 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314802/ [7314805] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:15:10 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314805/ [7314808] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:13:23 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314808/ [7314811] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/redirect} time: 00:14:57 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314811/ [7314814] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:11:56 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314814/ [7314816] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:39:18 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314816/ [7314819] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_rgw_multisite} time: 00:16:33 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314819/ [7314822] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:34:26 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314822/ [7314825] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-bitmap} supported-random-distro$/{ubuntu_20.04} tasks/failover} time: 00:13:27 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314825/ [7314828] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:27:18 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314828/ [7314831] rados/standalone/{supported-random-distro$/{rhel_8} workloads/crush} time: 00:26:22 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314831/ [7314835] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:19:12 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314835/ [7314838] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:13:45 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314838/ [7314841] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:15:31 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314841/ [7314844] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:17:43 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314844/ [7314847] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/redirect_promote_tests} time: 00:22:04 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314847/ [7314850] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/cache-snaps} time: 00:35:35 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314850/ [7314853] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} time: 00:47:24 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314853/ [7314856] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/rados_python} time: 00:21:45 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314856/ [7314859] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:15:04 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314859/ [7314862] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:17:38 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314862/ [7314865] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_set_mon_crush_locations} time: 00:29:16 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314865/ [7314868] rados/singleton-nomsgr/{all/crushdiff mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:10:01 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314868/ [7314871] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/redirect_set_object} time: 00:15:58 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314871/ [7314874] rados/objectstore/{backends/fusestore supported-random-distro$/{ubuntu_latest}} time: 00:11:02 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314874/ [7314877] rados/singleton/{all/mon-config mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04}} time: 00:11:07 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314877/ [7314880] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/pool-create-delete} time: 00:21:07 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314880/ [7314883] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:21:27 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314883/ [7314886] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} tasks/mon_recovery} time: 00:23:50 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314886/ [7314889] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:10:41 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314889/ [7314892] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_adoption} time: 00:15:36 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314892/ [7314895] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/set-chunks-read} time: 00:20:46 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314895/ [7314898] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:15:00 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314898/ [7314901] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:22:35 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314901/ [7314904] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{ubuntu_20.04} tasks/insights} time: 00:13:23 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314904/ [7314907] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:19:04 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314907/ [7314910] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/rados_stress_watch} time: 00:22:13 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314910/ [7314913] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:12:36 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314913/ [7314916] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:36:13 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314916/ [7314919] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:24:42 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314919/ [7314922] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:24:17 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314922/ [7314925] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/small-objects-balanced} time: 00:28:02 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314925/ [7314928] rados/standalone/{supported-random-distro$/{rhel_8} workloads/erasure-code} time: 01:28:32 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314928/ [7314932] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:15 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314932/ [7314935] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_cephadm} time: 00:18:50 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314935/ [7314937] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:30:31 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314937/ [7314941] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:35:08 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314941/ [7314944] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:17:32 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314944/ [7314947] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:10:46 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314947/ [7314949] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep} time: 00:21:19 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314949/ [7314952] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} thrashers/none thrashosds-health workloads/small-objects-localized} time: 00:20:05 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314952/ [7314955] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:19:05 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314955/ [7314958] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:17:46 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314958/ [7314961] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:14:28 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314961/ [7314964] rados/singleton/{all/peer mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:14:55 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314964/ [7314968] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:17:21 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314968/ [7314974] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:15:12 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314974/ [7314976] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/radosbench} time: 01:21:38 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314976/ [7314978] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/small-objects} time: 00:23:47 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314978/ [7314980] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/sync workloads/rados_5925} time: 00:18:35 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314980/ [7314982] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_cephadm_repos} time: 00:08:42 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314982/ [7314984] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} tasks/rados_striper} time: 00:13:14 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314984/ [7314987] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:22:12 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314987/ [7314990] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04} tasks/mon_clock_no_skews} time: 00:08:47 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314990/ [7314993] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:13:41 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314993/ [7314995] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{ubuntu_latest} tasks/module_selftest} time: 00:29:05 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314995/ [7314999] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/snaps-few-objects-balanced} time: 00:37:47 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7314999/ [7315003] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{centos_8}} time: 00:15:49 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315003/ [7315006] rados/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_20.04} 1-start 2-services/rgw 3-final} time: 00:19:10 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315006/ [7315009] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04}} time: 00:14:49 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315009/ [7315012] rados/standalone/{supported-random-distro$/{centos_8} workloads/mgr} time: 00:18:15 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315012/ [7315015] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:17:45 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315015/ [7315018] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-radosbench} time: 01:38:22 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315018/ [7315021] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:16:59 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315021/ [7315024] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:12:56 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315024/ [7315027] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:47 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315027/ [7315030] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/snaps-few-objects-localized} time: 00:28:06 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315030/ [7315033] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04}} time: 00:09:26 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315033/ [7315035] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:17:19 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315035/ [7315036] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:15:50 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315036/ [7315038] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:22:41 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315038/ [7315040] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:25:40 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315040/ [7315042] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} time: 00:32:36 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315042/ [7315044] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:31:46 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315044/ [7315046] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:14:53 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315046/ [7315048] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:31:26 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315048/ [7315050] rados/singleton/{all/radostool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:21:38 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315050/ [7315052] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_big} time: 00:33:50 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315052/ [7315054] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:24:27 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315054/ [7315056] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:23:47 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315056/ [7315057] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_nfs} time: 00:33:37 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315057/ [7315058] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/rados_api_tests} time: 00:30:14 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315058/ [7315059] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:34:21 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315059/ [7315060] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/write_fadvise_dontneed} time: 00:16:42 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315060/ [7315061] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-stupid} supported-random-distro$/{centos_8} tasks/per_module_finisher_stats} time: 00:21:33 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315061/ [7315062] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:24:36 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315062/ [7315063] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:11:33 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315063/ [7315064] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:19:18 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315064/ [7315065] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:20:11 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315065/ [7315066] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04}} time: 00:15:01 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315066/ [7315067] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/rbd_cls} time: 00:22:11 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315067/ [7315068] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/misc} time: 01:10:50 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315068/ [7315069] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/admin_socket_objecter_requests} time: 00:21:47 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315069/ [7315070] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:15:30 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315070/ [7315071] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:11:33 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315071/ [7315072] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-small-objects-balanced} time: 00:23:59 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315072/ [7315073] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:36 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315073/ [7315074] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:24:42 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315074/ [7315075] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_mix} time: 00:23:01 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315075/ [7315076] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{ubuntu_latest}} time: 00:38:46 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315076/ [7315077] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_orch_cli} time: 00:19:02 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315077/ [7315078] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache-agent-big} time: 00:36:08 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315078/ [7315080] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:30:12 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315080/ [7315081] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} time: 00:40:39 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315081/ [7315082] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:15:36 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315082/ [7315083] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:15:22 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315083/ [7315084] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:15:33 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315084/ [7315085] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:14:25 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315085/ [7315086] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-agent-small} time: 00:15:13 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315086/ [7315087] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04}} time: 00:33:37 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315087/ [7315088] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-hybrid} supported-random-distro$/{rhel_8} tasks/progress} time: 00:27:59 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315088/ [7315089] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_orch_cli_mon} time: 00:38:28 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315089/ [7315091] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:12:25 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315091/ [7315092] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:24:34 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315092/ [7315093] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/many workloads/rados_mon_osdmap_prune} time: 01:06:42 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315093/ [7315097] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:15:34 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315097/ [7315099] rados/singleton/{all/test-noautoscale-flag mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:12:28 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315099/ [7315101] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:16:59 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315101/ [7315103] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} tasks/rados_workunit_loadgen_mostlyread} time: 00:30:22 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315103/ [7315105] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-bitmap} tasks/dashboard} time: 01:09:07 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315105/ [7315107] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:30:22 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315107/ [7315111] rados/upgrade/parallel/{0-random-distro$/{ubuntu_20.04} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:35:05 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315111/ [7315113] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_rgw_multisite} time: 00:19:06 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315113/ [7315115] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps} time: 00:27:18 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315115/ [7315117] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:13:24 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315117/ [7315121] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:12:23 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315121/ [7315124] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-small-objects-fast-read} time: 00:27:43 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315124/ [7315126] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:46 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315126/ [7315128] rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_rhel8} 1-start 2-services/basic 3-final} time: 00:17:53 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315128/ [7315130] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/quincy backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} time: 00:40:25 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315130/ [7315132] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:19:05 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315132/ [7315134] rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:53:10 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315134/ [7315136] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-snaps-balanced} time: 00:23:06 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315136/ [7315138] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:15:07 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315138/ [7315140] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:00 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315140/ [7315142] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:11:59 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315142/ [7315144] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} time: 02:05:50 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315144/ [7315146] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_20.04} tasks/prometheus} time: 00:12:08 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315146/ [7315148] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:41:26 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315148/ [7315150] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:19:17 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315150/ [7315152] rados/objectstore/{backends/objectstore-bluestore-a supported-random-distro$/{centos_8}} time: 02:18:27 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315152/ [7315154] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache-snaps} time: 00:21:57 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315154/ [7315156] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/readwrite} time: 00:15:10 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315156/ [7315158] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:11:31 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315158/ [7315160] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:15:33 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315160/ [7315162] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_set_mon_crush_locations} time: 00:22:35 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315162/ [7315164] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/one workloads/rados_mon_workunits} time: 00:30:39 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315164/ [7315166] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04}} time: 00:20:48 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315166/ [7315168] rados/standalone/{supported-random-distro$/{centos_8} workloads/mon} time: 01:10:25 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315168/ [7315170] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache} time: 00:14:36 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315170/ [7315172] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:11:35 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315172/ [7315174] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_adoption} time: 00:17:46 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315174/ [7315176] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:10:20 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315176/ [7315178] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites} time: 00:26:42 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315178/ [7315180] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:22:59 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315180/ [7315183] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/repave-all} time: 00:14:14 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315183/ [7315185] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/dedup-io-mixed} time: 00:13:29 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315185/ [7315187] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:41 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315187/ [7315189] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:31:56 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315189/ [7315191] rados/singleton-nomsgr/{all/crushdiff mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:12:16 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315191/ [7315193] rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:14:51 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315193/ [7315196] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} tasks/repair_test} time: 00:21:38 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315196/ [7315198] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:35:01 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315198/ [7315200] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:15:17 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315200/ [7315203] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{ubuntu_20.04} tasks/workunits} time: 00:10:07 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315203/ [7315205] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/dedup-io-snaps} time: 00:17:02 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315205/ [7315207] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:19:23 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315207/ [7315209] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:10:28 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315209/ [7315211] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:23:24 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315211/ [7315213] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} time: 00:17:34 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315213/ [7315217] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_cephadm} time: 00:20:59 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315217/ [7315219] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/test_rbd_api} time: 00:25:54 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315219/ [7315221] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/pool-snaps-few-objects} time: 00:28:35 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315221/ [7315223] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:25:26 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315223/ [7315225] rados/singleton/{all/admin-socket mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:11:31 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315225/ [7315227] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:09:31 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315227/ [7315229] rados/standalone/{supported-random-distro$/{rhel_8} workloads/osd-backfill} time: 02:35:00 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315229/ [7315231] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:36:20 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315231/ [7315233] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/pool-create-delete} time: 00:20:53 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315233/ [7315235] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:18:00 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315235/ [7315237] rados/objectstore/{backends/objectstore-bluestore-b supported-random-distro$/{rhel_8}} time: 00:15:19 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315237/ [7315239] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:13:36 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315239/ [7315241] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/rados_api_tests} time: 00:33:02 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315241/ [7315243] rados/singleton/{all/backfill-toofull mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:24:32 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315243/ [7315245] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:18:42 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315245/ [7315247] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:13:29 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315247/ [7315249] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} tasks/scrub_test} time: 00:15:17 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315249/ [7315251] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_cephadm_repos} time: 00:07:14 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315251/ [7315253] rados/singleton/{all/deduptool mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:12:31 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315253/ [7315255] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/radosbench-high-concurrency} time: 00:25:44 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315255/ [7315257] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:07 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315257/ [7315259] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects} time: 00:26:15 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315259/ [7315263] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{rhel_8} tasks/crash} time: 00:25:18 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315263/ [7315265] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:10:20 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315265/ [7315267] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:19:07 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315267/ [7315269] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:29:01 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315269/ [7315271] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:15:46 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315271/ [7315273] rados/singleton/{all/divergent_priors mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:16:42 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315273/ [7315275] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:15:48 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315275/ [7315277] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:23:20 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315277/ [7315279] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/radosbench} time: 00:53:48 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315279/ [7315285] rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0} 1-start 2-services/rgw 3-final} time: 00:17:49 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315285/ [7315287] rados/singleton/{all/divergent_priors2 mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04}} time: 00:11:08 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315287/ [7315289] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:14:31 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315289/ [7315291] rados/standalone/{supported-random-distro$/{centos_8} workloads/osd} time: 02:39:39 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315291/ [7315293] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/libcephsqlite} time: 00:18:50 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315293/ [7315297] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/redirect} time: 00:19:22 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315297/ [7315299] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:17:40 info: http://pulpito.ceph.com/yuriw-2023-06-23_20:58:40-rados-reef-distro-default-smithi/7315299/