2023-12-05T19:00:26.753 INFO:root:teuthology version: 0.0.1.dev214+gf2de108 2023-12-05T19:00:26.753 INFO:teuthology.results:subset: '111/120000' 2023-12-05T19:00:26.753 INFO:teuthology.results:seed: '9131' 2023-12-05T21:11:02.950 INFO:root:teuthology version: 0.0.1.dev214+gf2de108 2023-12-05T21:11:02.973 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-12-06T00:56:24.427 INFO:teuthology.results:Tests finished! gathering results... 2023-12-06T00:56:49.776 INFO:scrape:Found 305 jobs 2023-12-06T00:56:50.673 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478131/teuthology.log 2023-12-06T00:56:52.913 INFO:scrape:Found 10 distinct failure reasons 2023-12-06T00:56:52.913 INFO:scrape:Failure: expected valgrind issues and found none 2023-12-06T00:56:52.913 INFO:scrape:3 jobs: ['7478064', '7478173', '7478175'] 2023-12-06T00:56:52.913 INFO:scrape:suites intersection: [] 2023-12-06T00:56:52.914 INFO:scrape:suites union: ['2-inject-leak/mon', '2-inject-leak/none', 'centos_latest}', 'ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async-v2only', 'objectstore/bluestore-comp-snappy', 'openstack}', 'rados', 'rados/valgrind-leaks/{1-start', 'rados/verify/{centos_latest', 'tasks/mon_recovery', 'validater/valgrind}'] 2023-12-06T00:56:52.914 INFO:scrape: 2023-12-06T00:56:52.914 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi060 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=244b703b22e9d7c48e37291bfeaf4b15a97cc628 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2023-12-06T00:56:52.914 INFO:scrape:2 jobs: ['7478136', '7478427'] 2023-12-06T00:56:52.914 INFO:scrape:suites intersection: ['debug/mgr', 'rados/dashboard/{0-single-container-host', 'tasks/e2e}'] 2023-12-06T00:56:52.915 INFO:scrape:suites union: ['debug/mgr', 'mon_election/classic', 'mon_election/connectivity', 'rados/dashboard/{0-single-container-host', 'random-objectstore$/{bluestore-comp-zlib}', 'random-objectstore$/{bluestore-stupid}', 'tasks/e2e}'] 2023-12-06T00:56:52.915 INFO:scrape: 2023-12-06T00:56:52.915 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/cls/test_cls_rgw.sh 2023-12-06T00:56:52.915 INFO:scrape:1 jobs: ['7478026'] 2023-12-06T00:56:52.916 INFO:scrape:suites: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async-v1only', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_cls_all', 'validater/valgrind}'] 2023-12-06T00:56:52.916 INFO:scrape: 2023-12-06T00:56:52.916 INFO:scrape:Dead: None 2023-12-06T00:56:52.916 INFO:scrape:1 jobs: ['7478131'] 2023-12-06T00:56:52.916 INFO:scrape: 2023-12-06T00:56:52.916 INFO:scrape:Failure: "2023-12-05T21:36:24.405335+0000 mon.a (mon.0) 802 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log 2023-12-06T00:56:52.916 INFO:scrape:2 jobs: ['7478449', '7478097'] 2023-12-06T00:56:52.916 INFO:scrape:suites intersection: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_api_tests', 'validater/valgrind}'] 2023-12-06T00:56:52.916 INFO:scrape:suites union: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_api_tests', 'validater/valgrind}'] 2023-12-06T00:56:52.917 INFO:scrape: 2023-12-06T00:56:52.917 INFO:scrape:Failure: Command failed (workunit test post-file.sh) on smithi062 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=244b703b22e9d7c48e37291bfeaf4b15a97cc628 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/post-file.sh' 2023-12-06T00:56:52.917 INFO:scrape:2 jobs: ['7478094', '7478343'] 2023-12-06T00:56:52.917 INFO:scrape:suites intersection: ['rados', 'rados/singleton-nomsgr/{all/ceph-post-file'] 2023-12-06T00:56:52.917 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados', 'rados/singleton-nomsgr/{all/ceph-post-file', 'supported-random-distro$/{centos_latest}}', 'supported-random-distro$/{ubuntu_20.04}}'] 2023-12-06T00:56:52.917 INFO:scrape: 2023-12-06T00:56:52.917 INFO:scrape:Failure: Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --balance-reads --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op copy_from 50 --op write_excl 50 --pool unique_pool_0' 2023-12-06T00:56:52.917 INFO:scrape:1 jobs: ['7478147'] 2023-12-06T00:56:52.917 INFO:scrape:suites: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{more-partial-recovery}', '3-scrub-overrides/{default}', 'backoff/normal', 'ceph', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'd-balancer/crush-compat', 'mon_election/classic', 'msgr-failures/osd-dispatch-delay', 'msgr/async-v1only', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'supported-random-distro$/{centos_latest}', 'thrashers/careful', 'thrashosds-health', 'workloads/snaps-few-objects-balanced}'] 2023-12-06T00:56:52.918 INFO:scrape: 2023-12-06T00:56:52.918 INFO:scrape:Failure: Command failed on smithi155 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:244b703b22e9d7c48e37291bfeaf4b15a97cc628 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5df5508-93a3-11ee-95a2-87774f69a715 -- ceph mon dump -f json' 2023-12-06T00:56:52.918 INFO:scrape:1 jobs: ['7478024'] 2023-12-06T00:56:52.918 INFO:scrape:suites: ['0-nvme-loop', 'agent/on', 'fixed-2', 'mon_election/classic', 'rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8', 'start}'] 2023-12-06T00:56:52.918 INFO:scrape: 2023-12-06T00:56:52.918 INFO:scrape:Failure: Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_lazy_omap_stats' 2023-12-06T00:56:52.918 INFO:scrape:1 jobs: ['7478447'] 2023-12-06T00:56:52.921 INFO:scrape:suites: ['mon_election/connectivity', 'rados', 'rados/singleton-nomsgr/{all/lazy_omap_stats_output', 'supported-random-distro$/{centos_latest}}'] 2023-12-06T00:56:52.921 INFO:scrape: 2023-12-06T00:56:52.921 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/cls/test_cls_rbd.sh 2023-12-06T00:56:52.921 INFO:scrape:1 jobs: ['7478311'] 2023-12-06T00:56:52.921 INFO:scrape:suites: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async-v1only', 'objectstore/bluestore-comp-zstd', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_cls_all', 'validater/valgrind}'] 2023-12-06T00:56:52.922 INFO:scrape: 2023-12-06T00:56:52.932 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi ================================================================= info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/ failed: 14 dead: 1 running: 0 waiting: 0 queued: 0 passed: 290 Fail ================================================================= [7478024] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:15:34 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478024/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478024/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2783ea9742824da7ab84dc4028171d83 Command failed on smithi155 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay- quay.apps.os.sepia.ceph.com/ceph- ci/ceph:244b703b22e9d7c48e37291bfeaf4b15a97cc628 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a5df5508-93a3-11ee-95a2-87774f69a715 -- ceph mon dump -f json' [7478026] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados tasks/rados_cls_all validater/valgrind} ----------------------------------------------------------------- time: 04:19:56 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478026/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478026/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d38ee05e9fd1441ba494421e99f4203f Command failed (workunit test cls/test_cls_rgw.sh) on smithi146 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=244b703b22e9d7c48e37291bfeaf4b15a97cc628 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh' [7478064] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} ----------------------------------------------------------------- time: 00:30:07 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478064/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478064/ expected valgrind issues and found none [7478094] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} ----------------------------------------------------------------- time: 00:08:56 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478094/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478094/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b014a288fa0449a0b57e0a31dca63e88 Command failed (workunit test post-file.sh) on smithi062 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=244b703b22e9d7c48e37291bfeaf4b15a97cc628 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/post-file.sh' [7478097] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} ----------------------------------------------------------------- time: 00:52:29 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478097/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478097/ "2023-12-05T20:01:10.725976+0000 mon.a (mon.0) 958 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [7478136] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-stupid} tasks/e2e} ----------------------------------------------------------------- time: 00:31:33 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478136/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478136/ sentry: https://sentry.ceph.com/organizations/ceph/?query=70849d071a83480294bfe090169d7f83 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi060 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=244b703b22e9d7c48e37291bfeaf4b15a97cc628 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [7478147] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/snaps-few-objects-balanced} ----------------------------------------------------------------- time: 00:34:58 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478147/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478147/ Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --balance-reads --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride- size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op copy_from 50 --op write_excl 50 --pool unique_pool_0' [7478173] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} ----------------------------------------------------------------- time: 00:20:21 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478173/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478173/ saw valgrind issues [7478175] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} ----------------------------------------------------------------- time: 00:43:59 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478175/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478175/ saw valgrind issues [7478311] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} ----------------------------------------------------------------- time: 04:13:59 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478311/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478311/ sentry: https://sentry.ceph.com/organizations/ceph/?query=df5fc2c2f858447098ba37dae40b9d80 Command failed (workunit test cls/test_cls_rbd.sh) on smithi037 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=244b703b22e9d7c48e37291bfeaf4b15a97cc628 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh' [7478343] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{centos_latest}} ----------------------------------------------------------------- time: 00:10:36 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478343/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478343/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8195ab1b167345d08fb040d36101f1f0 Command failed (workunit test post-file.sh) on smithi053 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=244b703b22e9d7c48e37291bfeaf4b15a97cc628 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/post-file.sh' [7478427] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-zlib} tasks/e2e} ----------------------------------------------------------------- time: 00:30:38 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478427/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478427/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f685609306374ae385e78a89c1fed850 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi027 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=244b703b22e9d7c48e37291bfeaf4b15a97cc628 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [7478447] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{centos_latest}} ----------------------------------------------------------------- time: 00:09:52 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478447/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478447/ sentry: https://sentry.ceph.com/organizations/ceph/?query=505e48ede0944029a14f50cfb1999a7c Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_lazy_omap_stats' [7478449] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} ----------------------------------------------------------------- time: 00:58:03 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478449/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478449/ "2023-12-05T21:36:24.405335+0000 mon.a (mon.0) 802 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log Dead ================================================================= [7478131] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478131/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478131/ Error reimaging machines: reached maximum tries (101) after waiting for 600 seconds Pass ================================================================= [7478010] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:18:13 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478010/ [7478011] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:20:57 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478011/ [7478012] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} time: 00:18:10 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478012/ [7478013] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:31:19 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478013/ [7478014] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:15:41 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478014/ [7478015] rados/objectstore/{backends/objectstore-memstore supported-random-distro$/{centos_8}} time: 00:14:35 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478015/ [7478016] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:32:17 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478016/ [7478017] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-small-objects} time: 00:36:28 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478017/ [7478018] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_cephadm_repos} time: 00:15:31 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478018/ [7478019] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:19 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478019/ [7478020] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04} thrashers/none thrashosds-health workloads/cache} time: 00:19:42 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478020/ [7478021] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:22:50 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478021/ [7478022] rados/singleton/{all/ec-inconsistent-hinfo mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:22:58 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478022/ [7478023] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} tasks/libcephsqlite} time: 00:23:51 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478023/ [7478025] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:24:04 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478025/ [7478027] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:16:31 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478027/ [7478028] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:24:25 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478028/ [7478029] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/scrub} time: 02:53:11 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478029/ [7478030] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{ubuntu_latest} tasks/workunits} time: 00:18:13 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478030/ [7478031] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:37:16 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478031/ [7478032] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:27:55 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478032/ [7478033] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/dedup-io-mixed} time: 00:17:04 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478033/ [7478034] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:36:51 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478034/ [7478035] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:15:12 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478035/ [7478036] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/many workloads/rados_mon_workunits} time: 00:38:55 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478036/ [7478037] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:22:33 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478037/ [7478038] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}} time: 00:10:30 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478038/ [7478039] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/dedup-io-snaps} time: 00:24:11 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478039/ [7478040] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} tasks/mon_clock_no_skews} time: 00:16:56 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478040/ [7478041] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:32:15 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478041/ [7478042] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:32:06 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478042/ [7478043] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:38:50 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478043/ [7478044] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:20:53 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478044/ [7478045] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/rados_api_tests} time: 00:39:05 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478045/ [7478046] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/pool-snaps-few-objects} time: 00:39:19 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478046/ [7478047] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_orch_cli} time: 00:26:22 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478047/ [7478048] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/test_rbd_api} time: 00:29:29 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478048/ [7478049] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:20:17 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478049/ [7478050] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:33:54 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478050/ [7478051] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04}} time: 00:40:44 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478051/ [7478052] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:35:04 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478052/ [7478053] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest}} time: 00:33:15 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478053/ [7478054] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:18:30 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478054/ [7478055] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:38:15 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478055/ [7478056] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/rados_api_tests} time: 00:41:25 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478056/ [7478057] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-snappy} tasks/dashboard} time: 01:18:22 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478057/ [7478058] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{rhel_8} tasks/crash} time: 00:24:18 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478058/ [7478059] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{centos_latest}} time: 00:40:04 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478059/ [7478060] rados/rest/{mgr-restful supported-random-distro$/{centos_8}} time: 00:19:28 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478060/ [7478061] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:25:10 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478061/ [7478062] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/c2c} time: 00:22:47 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478062/ [7478063] rados/upgrade/parallel/{0-random-distro$/{ubuntu_20.04} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:49:23 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478063/ [7478065] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados tasks/mon_recovery validater/lockdep} time: 00:21:14 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478065/ [7478066] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_orch_cli_mon} time: 00:33:49 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478066/ [7478067] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:23:46 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478067/ [7478068] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:15:05 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478068/ [7478069] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/radosbench-high-concurrency} time: 00:43:19 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478069/ [7478070] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:14:01 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478070/ [7478071] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:22:25 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478071/ [7478072] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:17:43 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478072/ [7478073] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/one workloads/snaps-few-objects} time: 00:45:13 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478073/ [7478074] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest}} time: 00:20:39 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478074/ [7478075] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04} tasks/rados_cls_all} time: 00:29:37 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478075/ [7478076] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools} 1-start 2-services/basic 3-final} time: 00:20:26 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478076/ [7478077] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:19:47 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478077/ [7478078] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/radosbench} time: 00:36:48 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478078/ [7478079] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:33:13 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478079/ [7478080] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:18:13 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478080/ [7478081] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:26:05 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478081/ [7478082] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:21:35 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478082/ [7478083] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/redirect} time: 00:19:58 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478083/ [7478084] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:17:58 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478084/ [7478085] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:33:24 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478085/ [7478086] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_rgw_multisite} time: 00:20:43 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478086/ [7478087] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:19:28 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478087/ [7478088] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:29:07 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478088/ [7478089] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{rhel_8} tasks/failover} time: 00:21:35 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478089/ [7478090] rados/standalone/{supported-random-distro$/{rhel_8} workloads/crush} time: 00:30:22 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478090/ [7478091] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:12:23 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478091/ [7478092] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:13:41 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478092/ [7478093] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:15:01 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478093/ [7478095] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/redirect_promote_tests} time: 00:21:02 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478095/ [7478096] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/cache-snaps} time: 00:30:22 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478096/ [7478098] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} tasks/rados_python} time: 00:16:28 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478098/ [7478099] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}} time: 00:12:00 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478099/ [7478100] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:34 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478100/ [7478101] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_set_mon_crush_locations} time: 00:26:11 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478101/ [7478102] rados/singleton-nomsgr/{all/crushdiff mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:13:36 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478102/ [7478103] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/redirect_set_object} time: 00:19:19 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478103/ [7478104] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:19:31 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478104/ [7478105] rados/objectstore/{backends/fusestore supported-random-distro$/{ubuntu_latest}} time: 00:10:37 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478105/ [7478106] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} thrashers/sync-many workloads/pool-create-delete} time: 00:15:55 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478106/ [7478107] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:20:08 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478107/ [7478108] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:27:01 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478108/ [7478109] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:09:26 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478109/ [7478110] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_adoption} time: 00:19:20 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478110/ [7478111] rados/singleton/{all/mon-config mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:13:50 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478111/ [7478112] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/set-chunks-read} time: 00:25:54 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478112/ [7478113] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:15:05 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478113/ [7478114] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{ubuntu_20.04} tasks/insights} time: 00:13:03 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478114/ [7478115] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:20:56 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478115/ [7478116] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} tasks/rados_stress_watch} time: 00:18:45 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478116/ [7478117] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:19:20 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478117/ [7478118] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:30:12 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478118/ [7478119] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:21:31 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478119/ [7478120] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:21:40 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478120/ [7478121] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/small-objects-balanced} time: 00:21:24 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478121/ [7478122] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/erasure-code} time: 01:27:19 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478122/ [7478123] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:50 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478123/ [7478124] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_ca_signed_key} time: 00:16:21 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478124/ [7478125] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:26:58 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478125/ [7478126] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:42:46 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478126/ [7478127] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:30:16 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478127/ [7478128] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:11:23 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478128/ [7478129] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep} time: 00:18:43 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478129/ [7478130] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/small-objects-localized} time: 00:22:06 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478130/ [7478132] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:19:40 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478132/ [7478133] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:15:08 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478133/ [7478134] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:24:14 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478134/ [7478135] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:21:08 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478135/ [7478137] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:19:42 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478137/ [7478138] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/radosbench} time: 01:28:19 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478138/ [7478139] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/small-objects} time: 00:23:03 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478139/ [7478140] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/sync workloads/rados_5925} time: 00:15:27 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478140/ [7478141] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_cephadm} time: 00:17:38 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478141/ [7478142] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} tasks/rados_striper} time: 00:10:38 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478142/ [7478143] rados/singleton/{all/peer mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:13:12 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478143/ [7478144] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:11:06 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478144/ [7478145] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:14:23 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478145/ [7478146] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{centos_8} tasks/module_selftest} time: 00:29:45 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478146/ [7478148] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{ubuntu_20.04}} time: 00:17:02 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478148/ [7478149] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools} 1-start 2-services/rgw 3-final} time: 00:14:08 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478149/ [7478150] rados/standalone/{supported-random-distro$/{centos_latest} workloads/mgr} time: 00:16:38 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478150/ [7478151] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest}} time: 00:17:37 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478151/ [7478153] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:22:39 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478153/ [7478155] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-radosbench} time: 02:16:46 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478155/ [7478157] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:19:27 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478157/ [7478159] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:13:17 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478159/ [7478161] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:31 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478161/ [7478163] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/snaps-few-objects-localized} time: 00:36:02 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478163/ [7478165] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:14:07 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478165/ [7478167] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04}} time: 00:17:46 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478167/ [7478169] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_cephadm_repos} time: 00:11:00 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478169/ [7478171] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:21:08 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478171/ [7478177] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:34:20 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478177/ [7478179] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:28 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478179/ [7478181] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:29:46 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478181/ [7478183] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_big} time: 00:36:01 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478183/ [7478185] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:24:18 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478185/ [7478187] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}} time: 00:09:14 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478187/ [7478189] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:24:02 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478189/ [7478191] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:18:03 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478191/ [7478193] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} thrashers/force-sync-many workloads/rados_api_tests} time: 00:28:58 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478193/ [7478195] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/write_fadvise_dontneed} time: 00:23:31 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478195/ [7478197] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-hybrid} supported-random-distro$/{rhel_8} tasks/per_module_finisher_stats} time: 00:19:38 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478197/ [7478199] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:24:46 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478199/ [7478201] rados/singleton/{all/radostool mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 00:28:07 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478201/ [7478203] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:10:39 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478203/ [7478205] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:28:52 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478205/ [7478207] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:19:28 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478207/ [7478209] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/rbd_cls} time: 00:21:56 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478209/ [7478211] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/misc} time: 01:06:45 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478211/ [7478213] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/admin_socket_objecter_requests} time: 00:19:20 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478213/ [7478215] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:29:46 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478215/ [7478217] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:16:00 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478217/ [7478219] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:10:05 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478219/ [7478221] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-small-objects-balanced} time: 00:30:11 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478221/ [7478223] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:28 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478223/ [7478225] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} tasks/rados_workunit_loadgen_mix} time: 00:22:44 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478225/ [7478227] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{ubuntu_20.04}} time: 00:31:24 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478227/ [7478229] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_orch_cli} time: 00:23:19 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478229/ [7478231] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/cache-agent-big} time: 00:23:43 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478231/ [7478233] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:24:15 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478233/ [7478235] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:33:02 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478235/ [7478237] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:31:27 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478237/ [7478239] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} time: 00:28:31 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478239/ [7478241] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:59 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478241/ [7478243] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:15:24 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478243/ [7478245] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:14:19 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478245/ [7478247] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:29:30 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478247/ [7478249] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/cache-agent-small} time: 00:25:42 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478249/ [7478251] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:36:15 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478251/ [7478253] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_20.04} tasks/progress} time: 00:24:50 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478253/ [7478255] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_orch_cli_mon} time: 00:38:05 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478255/ [7478257] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:12:13 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478257/ [7478259] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:25:59 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478259/ [7478261] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/many workloads/rados_mon_osdmap_prune} time: 01:07:08 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478261/ [7478263] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:14:45 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478263/ [7478265] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:28:45 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478265/ [7478267] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:15:28 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478267/ [7478269] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:21:05 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478269/ [7478271] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mostlyread} time: 00:35:02 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478271/ [7478273] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-stupid} tasks/dashboard} time: 01:08:10 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478273/ [7478275] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:18:06 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478275/ [7478277] rados/standalone/{supported-random-distro$/{centos_8} workloads/mon-stretch} time: 00:28:13 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478277/ [7478279] rados/upgrade/parallel/{0-random-distro$/{ubuntu_20.04} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:41:46 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478279/ [7478281] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_rgw_multisite} time: 00:18:24 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478281/ [7478283] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:14:13 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478283/ [7478285] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps} time: 00:28:35 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478285/ [7478287] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:13:33 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478287/ [7478289] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:19:39 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478289/ [7478291] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-fast-read} time: 00:21:04 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478291/ [7478293] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:58 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478293/ [7478295] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools} 1-start 2-services/basic 3-final} time: 00:12:49 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478295/ [7478297] rados/singleton/{all/test-noautoscale-flag mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:18:34 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478297/ [7478299] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/quincy backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} time: 00:37:16 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478299/ [7478301] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:12:22 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478301/ [7478303] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/cache-snaps-balanced} time: 00:28:05 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478303/ [7478305] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:15:24 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478305/ [7478307] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:14:49 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478307/ [7478309] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:11:19 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478309/ [7478313] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{centos_8} tasks/prometheus} time: 00:15:37 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478313/ [7478315] rados/singleton/{all/thrash-backfill-full mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04}} time: 01:14:44 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478315/ [7478317] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:20:08 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478317/ [7478319] rados/objectstore/{backends/objectstore-bluestore-a supported-random-distro$/{ubuntu_20.04}} time: 02:13:46 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478319/ [7478321] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache-snaps} time: 00:23:38 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478321/ [7478323] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} tasks/readwrite} time: 00:14:20 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478323/ [7478325] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:10:18 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478325/ [7478327] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:16:14 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478327/ [7478329] rados/singleton/{all/thrash-eio mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:37:24 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478329/ [7478331] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_set_mon_crush_locations} time: 00:19:12 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478331/ [7478333] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/one workloads/rados_mon_workunits} time: 00:37:52 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478333/ [7478335] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/mon} time: 01:08:16 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478335/ [7478337] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache} time: 00:14:38 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478337/ [7478339] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:17:49 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478339/ [7478341] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_adoption} time: 00:19:28 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478341/ [7478345] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest}} time: 00:21:39 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478345/ [7478347] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites} time: 00:21:48 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478347/ [7478349] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/repave-all} time: 00:13:51 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478349/ [7478351] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/dedup-io-mixed} time: 00:14:38 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478351/ [7478353] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:35:51 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478353/ [7478355] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:22:35 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478355/ [7478357] rados/singleton-nomsgr/{all/crushdiff mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:18:37 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478357/ [7478359] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:25:24 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478359/ [7478361] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:20:01 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478361/ [7478363] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/repair_test} time: 00:19:30 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478363/ [7478365] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:26:52 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478365/ [7478367] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:15:30 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478367/ [7478369] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-bitmap} supported-random-distro$/{rhel_8} tasks/workunits} time: 00:19:27 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478369/ [7478371] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/dedup-io-snaps} time: 00:27:02 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478371/ [7478373] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:26 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478373/ [7478375] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:12:46 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478375/ [7478377] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:19:51 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478377/ [7478379] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} time: 00:13:13 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478379/ [7478381] rados/singleton/{all/watch-notify-same-primary mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:14:07 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478381/ [7478383] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_ca_signed_key} time: 00:22:19 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478383/ [7478385] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/test_rbd_api} time: 00:20:15 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478385/ [7478387] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/pool-snaps-few-objects} time: 00:28:33 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478387/ [7478389] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:25:05 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478389/ [7478391] rados/singleton/{all/admin-socket mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest}} time: 00:08:43 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478391/ [7478393] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:09:44 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478393/ [7478395] rados/standalone/{supported-random-distro$/{centos_8} workloads/osd-backfill} time: 02:36:32 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478395/ [7478397] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:37:21 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478397/ [7478399] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/pool-create-delete} time: 00:26:47 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478399/ [7478401] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:20:36 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478401/ [7478403] rados/objectstore/{backends/objectstore-bluestore-b supported-random-distro$/{centos_8}} time: 00:11:15 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478403/ [7478405] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04} tasks/mon_clock_with_skews} time: 00:09:23 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478405/ [7478407] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/rados_api_tests} time: 00:32:18 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478407/ [7478409] rados/singleton/{all/backfill-toofull mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:26:30 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478409/ [7478411] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:11:32 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478411/ [7478413] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:13:45 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478413/ [7478415] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/scrub_test} time: 00:14:57 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478415/ [7478417] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_cephadm} time: 00:20:52 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478417/ [7478419] rados/singleton/{all/deduptool mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:13:57 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478419/ [7478421] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/radosbench-high-concurrency} time: 00:27:54 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478421/ [7478423] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:29:39 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478423/ [7478425] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/ec-small-objects} time: 00:22:39 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478425/ [7478429] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{rhel_8} tasks/crash} time: 00:22:06 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478429/ [7478431] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:11:55 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478431/ [7478433] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_cephadm_repos} time: 00:10:46 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478433/ [7478435] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:29:07 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478435/ [7478437] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:21:20 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478437/ [7478439] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:15:42 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478439/ [7478441] rados/singleton/{all/divergent_priors mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:13:19 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478441/ [7478443] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:15:25 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478443/ [7478445] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/radosbench} time: 00:52:18 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478445/ [7478451] rados/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_20.04} 1-start 2-services/rgw 3-final} time: 00:19:24 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478451/ [7478453] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:13:37 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478453/ [7478455] rados/singleton/{all/divergent_priors2 mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04}} time: 00:10:39 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478455/ [7478457] rados/standalone/{supported-random-distro$/{centos_8} workloads/osd} time: 02:39:07 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478457/ [7478459] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/libcephsqlite} time: 00:16:36 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478459/ [7478461] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:22:34 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478461/ [7478463] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/redirect} time: 00:21:46 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478463/ [7478465] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:19:01 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478465/ [7478467] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} thrashers/sync workloads/rados_5925} time: 00:12:25 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478467/ [7478469] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/cache-snaps} time: 00:31:10 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478469/ [7478471] rados/singleton/{all/dump-stuck mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:14:04 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478471/ [7478473] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} tasks/mon_recovery} time: 00:13:55 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478473/ [7478475] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:16:36 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478475/ [7478477] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:20:52 info: https://pulpito.ceph.com/yuriw-2023-12-05_18:59:03-rados-wip-yuri4-testing-2023-12-04-1129-reef-distro-default-smithi/7478477/