2024-03-09T11:56:22.750 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-03-09T11:56:22.750 INFO:teuthology.results:subset: '111/120000' 2024-03-09T11:56:22.750 INFO:teuthology.results:seed: '1270' 2024-03-10T00:38:10.226 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-03-10T00:38:10.247 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2024-03-10T10:09:04.908 INFO:teuthology.results:Tests finished! gathering results... 2024-03-10T10:09:30.029 INFO:scrape:Found 294 jobs 2024-03-10T10:09:30.298 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587281/teuthology.log 2024-03-10T10:09:38.889 INFO:scrape:Found 13 distinct failure reasons 2024-03-10T10:09:38.890 INFO:scrape:Failure: Command failed on smithi096 with status 5: 'sudo systemctl stop ceph-f8052b28-de2b-11ee-95c7-87774f69a715@mon.a' 2024-03-10T10:09:38.890 INFO:scrape:34 jobs 2024-03-10T10:09:38.890 INFO:scrape:suites intersection: [] 2024-03-10T10:09:38.891 INFO:scrape:suites union: ['0-nvme-loop', '0-size-min-size-overrides/2-size-2-min-size', '0-size-min-size-overrides/3-size-2-min-size', '1-install/nautilus', '1-install/nautilus-v1only', '1-install/nautilus-v2only', '1-install/octopus', '1-install/pacific', '1-install/quincy', '1-install/reef', '1-start', '2-ops/repave-all}', '2-ops/rm-zap-add}', '2-ops/rm-zap-flag}', '2-ops/rm-zap-wait}', '2-ops/rmdir-reactivate}', '2-services/basic', '2-services/rgw', '3-final}', 'agent/off', 'agent/on', 'backoff/normal', 'backoff/peering', 'backoff/peering_and_degraded', 'ceph', 'clusters/{openstack', 'd-balancer/crush-compat', 'd-balancer/on', 'debug/mgr', 'fixed-2', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/osd-delay', 'rados', 'rados/cephadm/osds/{0-distro/centos_9.stream', 'rados/cephadm/osds/{0-distro/centos_9.stream_runc', 'rados/cephadm/osds/{0-distro/ubuntu_22.04', 'rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream_runc}', 'rados/cephadm/smoke/{0-distro/centos_9.stream', 'rados/cephadm/smoke/{0-distro/centos_9.stream_runc', 'rados/cephadm/smoke/{0-distro/ubuntu_22.04', 'rados/cephadm/workunits/{0-distro/centos_9.stream', 'rados/cephadm/workunits/{0-distro/centos_9.stream_runc', 'rados/cephadm/workunits/{0-distro/ubuntu_22.04', 'rados/dashboard/{0-single-container-host', 'rados/singleton-nomsgr/{all/ceph-snapmapper', 'rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools}', 'random-objectstore$/{bluestore-low-osd-mem-target}', 'start}', 'supported-random-distro$/{centos_latest}}', 'supported-random-distro$/{ubuntu_latest}}', 'task/test_ca_signed_key}', 'task/test_extra_daemon_features}', 'task/test_host_drain}', 'task/test_iscsi_container/{centos_9.stream', 'task/test_monitoring_stack_basic}', 'task/test_rgw_multisite}', 'task/test_set_mon_crush_locations}', 'tasks/e2e}', 'test_iscsi_container}}', 'thrashers/careful', 'thrashers/default', 'thrashers/mapgap', 'thrashers/morepggrow', 'thrashers/none', 'thrashers/pggrow', 'thrashosds-health', 'three-plus-one}', 'workloads/cache-snaps}', 'workloads/radosbench}', 'workloads/rbd_cls}', 'workloads/snaps-few-objects}', 'workloads/test_rbd_api}'] 2024-03-10T10:09:38.891 INFO:scrape: 2024-03-10T10:09:38.891 INFO:scrape:Failure: valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun 2024-03-10T10:09:38.891 INFO:scrape:5 jobs: ['7587275', '7587012', '7587149', '7587278', '7587674'] 2024-03-10T10:09:38.891 INFO:scrape:suites intersection: [] 2024-03-10T10:09:38.891 INFO:scrape:suites union: ['2-inject-leak/none', 'centos_latest}', 'ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/valgrind-leaks/{1-start', 'rados/verify/{centos_latest', 'tasks/mon_recovery', 'tasks/rados_api_tests', 'tasks/rados_cls_all', 'validater/valgrind}'] 2024-03-10T10:09:38.891 INFO:scrape: 2024-03-10T10:09:38.891 INFO:scrape:Dead: None 2024-03-10T10:09:38.892 INFO:scrape:6 jobs: ['7587281', '7587183', '7587386', '7587059', '7587377', '7587066'] 2024-03-10T10:09:38.892 INFO:scrape: 2024-03-10T10:09:38.892 INFO:scrape:Failure: Command failed on smithi029 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7bbfe926-de63-11ee-95c7-87774f69a715 -e sha1=6b01fe95f3aa686f1951a0c5828907a52930b92a -- bash -c 'ceph orch ps'" 2024-03-10T10:09:38.892 INFO:scrape:2 jobs: ['7587434', '7587085'] 2024-03-10T10:09:38.892 INFO:scrape:suites intersection: ['0-start', '1-tasks', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'test_rbd_api', 'test_rbd_python}}', 'upgrade-sequence', 'workload/{ec-rados-default'] 2024-03-10T10:09:38.892 INFO:scrape:suites union: ['0-start', '1-tasks', 'mon_election/classic', 'mon_election/connectivity', 'rados/upgrade/parallel/{0-random-distro$/{centos_9.stream_runc}', 'rados/upgrade/parallel/{0-random-distro$/{centos_9.stream}', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'test_rbd_api', 'test_rbd_python}}', 'upgrade-sequence', 'workload/{ec-rados-default'] 2024-03-10T10:09:38.892 INFO:scrape: 2024-03-10T10:09:38.892 INFO:scrape:Failure: Command failed (workunit test mon/mkfs.sh) on smithi155 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6b01fe95f3aa686f1951a0c5828907a52930b92a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/mon/mkfs.sh' 2024-03-10T10:09:38.892 INFO:scrape:1 jobs: ['7587508'] 2024-03-10T10:09:38.892 INFO:scrape:suites: ['rados/standalone/{supported-random-distro$/{centos_latest}', 'workloads/mon}'] 2024-03-10T10:09:38.893 INFO:scrape: 2024-03-10T10:09:38.893 INFO:scrape:Failure: No module named 'tasks' 2024-03-10T10:09:38.893 INFO:scrape:1 jobs: ['7586982'] 2024-03-10T10:09:38.893 INFO:scrape:suites: ['1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{more-partial-recovery}', '3-scrub-overrides/{max-simultaneous-scrubs-1}', 'backoff/peering', 'ceph', 'clusters/{fixed-2', 'crc-failures/default', 'd-balancer/read', 'mon_election/connectivity', 'msgr-failures/osd-dispatch-delay', 'msgr/async-v2only', 'objectstore/bluestore-comp-zlib', 'openstack}', 'rados', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/default', 'thrashosds-health', 'workloads/cache}'] 2024-03-10T10:09:38.893 INFO:scrape: 2024-03-10T10:09:38.893 INFO:scrape:Failure: Command failed (workunit test cephadm/test_cephadm.sh) on smithi033 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6b01fe95f3aa686f1951a0c5828907a52930b92a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' 2024-03-10T10:09:38.893 INFO:scrape:1 jobs: ['7587045'] 2024-03-10T10:09:38.893 INFO:scrape:suites: ['agent/off', 'mon_election/classic', 'rados/cephadm/workunits/{0-distro/centos_9.stream', 'task/test_cephadm}'] 2024-03-10T10:09:38.893 INFO:scrape: 2024-03-10T10:09:38.893 INFO:scrape:Dead: b'2024-03-10T01:13:39.899 DEBUG:teuthology.exit:Finished running handlers' 2024-03-10T10:09:38.894 INFO:scrape:2 jobs: ['7587081', '7587428'] 2024-03-10T10:09:38.894 INFO:scrape:suites intersection: ['rados', 'rados/singleton-nomsgr/{all/admin_socket_output'] 2024-03-10T10:09:38.894 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados', 'rados/singleton-nomsgr/{all/admin_socket_output', 'supported-random-distro$/{centos_latest}}', 'supported-random-distro$/{ubuntu_latest}}'] 2024-03-10T10:09:38.894 INFO:scrape: 2024-03-10T10:09:38.894 INFO:scrape:Failure: Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_lazy_omap_stats' 2024-03-10T10:09:38.895 INFO:scrape:2 jobs: ['7587251', '7587680'] 2024-03-10T10:09:38.895 INFO:scrape:suites intersection: ['rados', 'rados/singleton-nomsgr/{all/lazy_omap_stats_output'] 2024-03-10T10:09:38.895 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados', 'rados/singleton-nomsgr/{all/lazy_omap_stats_output', 'supported-random-distro$/{centos_latest}}', 'supported-random-distro$/{ubuntu_latest}}'] 2024-03-10T10:09:38.895 INFO:scrape: 2024-03-10T10:09:38.895 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/cls/test_cls_2pc_queue.sh 2024-03-10T10:09:38.895 INFO:scrape:1 jobs: ['7587472'] 2024-03-10T10:09:38.895 INFO:scrape:suites: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async-v1only', 'objectstore/bluestore-comp-zstd', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_cls_all', 'validater/valgrind}'] 2024-03-10T10:09:38.895 INFO:scrape: 2024-03-10T10:09:38.895 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rados/test.sh 2024-03-10T10:09:38.896 INFO:scrape:1 jobs: ['7587032'] 2024-03-10T10:09:38.896 INFO:scrape:suites: ['1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{more-async-partial-recovery}', '3-scrub-overrides/{max-simultaneous-scrubs-5}', 'backoff/peering_and_degraded', 'ceph', 'clusters/{fixed-2', 'crc-failures/default', 'd-balancer/read', 'mon_election/connectivity', 'msgr-failures/osd-dispatch-delay', 'msgr/async', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/pggrow', 'thrashosds-health', 'workloads/rados_api_tests}'] 2024-03-10T10:09:38.896 INFO:scrape: 2024-03-10T10:09:38.896 INFO:scrape:Failure: reached maximum tries (51) after waiting for 300 seconds 2024-03-10T10:09:38.896 INFO:scrape:1 jobs: ['7587573'] 2024-03-10T10:09:38.896 INFO:scrape:suites: ['2-inject-leak/osd', 'centos_latest}', 'rados/valgrind-leaks/{1-start'] 2024-03-10T10:09:38.896 INFO:scrape: 2024-03-10T10:09:38.896 INFO:scrape:Failure: Command failed (workunit test scrub/osd-scrub-repair.sh) on smithi162 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6b01fe95f3aa686f1951a0c5828907a52930b92a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh' 2024-03-10T10:09:38.896 INFO:scrape:1 jobs: ['7587021'] 2024-03-10T10:09:38.896 INFO:scrape:suites: ['rados/standalone/{supported-random-distro$/{centos_latest}', 'workloads/scrub}'] 2024-03-10T10:09:38.897 INFO:scrape: 2024-03-10T10:09:38.908 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi ================================================================= info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/ failed: 50 dead: 8 running: 0 waiting: 0 queued: 0 passed: 236 Fail ================================================================= [7586982] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache} ----------------------------------------------------------------- time: 00:09:03 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7586982/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7586982/ sentry: https://sentry.ceph.com/organizations/ceph/?query=af48db8a4ebc4d19b823a01bacfe5948 No module named 'tasks' [7586998] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/quincy backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/test_rbd_api} ----------------------------------------------------------------- time: 00:10:16 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7586998/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7586998/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a01026cd5fe0490f91e0b15a2830fe1a Command failed on smithi042 with status 5: 'sudo systemctl stop ceph-e41a6440-de10-11ee-95c7-87774f69a715@mon.a' [7587000] rados/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-add} ----------------------------------------------------------------- time: 00:05:21 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587000/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587000/ sentry: https://sentry.ceph.com/organizations/ceph/?query=832f48423f2f43f7bad62b7770fb19be Command failed on smithi057 with status 5: 'sudo systemctl stop ceph-67b7bce0-de10-11ee-95c7-87774f69a715@mon.smithi057' [7587012] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados tasks/rados_cls_all validater/valgrind} ----------------------------------------------------------------- time: 01:41:37 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587012/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587012/ valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun [7587021] rados/standalone/{supported-random-distro$/{centos_latest} workloads/scrub} ----------------------------------------------------------------- time: 01:33:25 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587021/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587021/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f845ac637c2b487da7e3717e296b76dd Command failed (workunit test scrub/osd-scrub-repair.sh) on smithi162 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6b01fe95f3aa686f1951a0c5828907a52930b92a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub- repair.sh' [7587023] rados/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_ca_signed_key} ----------------------------------------------------------------- time: 00:10:52 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587023/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587023/ sentry: https://sentry.ceph.com/organizations/ceph/?query=556774e1c8fb4fbe9b1b213fc013a77b Command failed on smithi123 with status 5: 'sudo systemctl stop ceph- abfe816a-de13-11ee-95c7-87774f69a715@mon.a' [7587032] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/rados_api_tests} ----------------------------------------------------------------- time: 04:16:08 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587032/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587032/ sentry: https://sentry.ceph.com/organizations/ceph/?query=65e5e89aaa204db8bfc353ca29f4beab Command failed (workunit test rados/test.sh) on smithi062 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6b01fe95f3aa686f1951a0c5828907a52930b92a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [7587045] rados/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_cephadm} ----------------------------------------------------------------- time: 00:12:27 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587045/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587045/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f4d66184b5c14123a8a37ac78ccfc06d Command failed (workunit test cephadm/test_cephadm.sh) on smithi033 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6b01fe95f3aa686f1951a0c5828907a52930b92a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [7587069] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/reef backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/cache-snaps} ----------------------------------------------------------------- time: 00:10:59 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587069/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587069/ sentry: https://sentry.ceph.com/organizations/ceph/?query=91283a5acf44421c9a9f68df9b8779bf Command failed on smithi026 with status 5: 'sudo systemctl stop ceph-91412da6-de17-11ee-95c7-87774f69a715@mon.a' [7587071] rados/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-flag} ----------------------------------------------------------------- time: 00:05:23 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587071/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587071/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ba837de0714e4c9c8755ce313b692816 Command failed on smithi057 with status 5: 'sudo systemctl stop ceph-626a3540-de17-11ee-95c7-87774f69a715@mon.smithi057' [7587085] rados/upgrade/parallel/{0-random-distro$/{centos_9.stream} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} ----------------------------------------------------------------- time: 00:18:39 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587085/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587085/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ae656e7752e549d8ab8357264bad0a9a Command failed on smithi097 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1048b196-de18-11ee-95c7-87774f69a715 -e sha1=6b01fe95f3aa686f1951a0c5828907a52930b92a -- bash -c 'ceph orch ps'" [7587101] rados/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/off fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:05:36 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587101/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587101/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b14778592f41416f96f8aefc2b922d15 Command failed on smithi006 with status 5: 'sudo systemctl stop ceph-e6e58192-de19-11ee-95c7-87774f69a715@mon.a' [7587124] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream_runc} 1-start 2-services/basic 3-final} ----------------------------------------------------------------- time: 00:05:21 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587124/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587124/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b36fd45fac134092ad0ebc259a01b006 Command failed on smithi169 with status 5: 'sudo systemctl stop ceph-b87235c6-de23-11ee-95c7-87774f69a715@mon.smithi169' [7587147] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/radosbench} ----------------------------------------------------------------- time: 00:10:30 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587147/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587147/ sentry: https://sentry.ceph.com/organizations/ceph/?query=21c8f29ff43245989de462c7fd40d617 Command failed on smithi037 with status 5: 'sudo systemctl stop ceph-8b773726-de26-11ee-95c7-87774f69a715@mon.a' [7587149] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} ----------------------------------------------------------------- time: 01:11:33 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587149/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587149/ valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun [7587157] rados/singleton-nomsgr/{all/ceph-snapmapper mon_election/connectivity rados supported-random-distro$/{centos_latest}} ----------------------------------------------------------------- time: 00:07:47 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587157/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587157/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1242c229a0d2459c84fedc96a8ebb1e5 Command failed on smithi103 with status 1: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_snap_mapper' [7587167] rados/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-wait} ----------------------------------------------------------------- time: 00:09:03 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587167/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587167/ sentry: https://sentry.ceph.com/organizations/ceph/?query=96037907409844449adf0fa3e513600e Command failed on smithi106 with status 5: 'sudo systemctl stop ceph- df4ed8a8-de27-11ee-95c7-87774f69a715@mon.smithi106' [7587189] rados/cephadm/workunits/{0-distro/ubuntu_22.04 agent/off mon_election/classic task/test_extra_daemon_features} ----------------------------------------------------------------- time: 00:10:45 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587189/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587189/ sentry: https://sentry.ceph.com/organizations/ceph/?query=15cffb17da9b49eca6178f875769e16a Command failed on smithi112 with status 5: 'sudo systemctl stop ceph-b7bd7b58-de29-11ee-95c7-87774f69a715@mon.a' [7587215] rados/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rmdir-reactivate} ----------------------------------------------------------------- time: 00:05:46 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587215/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587215/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8f11603018294045894e05b4042af70f Command failed on smithi008 with status 5: 'sudo systemctl stop ceph-54685be8-de2b-11ee-95c7-87774f69a715@mon.smithi008' [7587217] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/rbd_cls} ----------------------------------------------------------------- time: 00:10:05 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587217/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587217/ sentry: https://sentry.ceph.com/organizations/ceph/?query=926aefa8619b49859949a579c471f317 Command failed on smithi096 with status 5: 'sudo systemctl stop ceph-f8052b28-de2b-11ee-95c7-87774f69a715@mon.a' [7587227] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-low-osd-mem-target} tasks/e2e} ----------------------------------------------------------------- time: 00:07:46 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587227/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587227/ sentry: https://sentry.ceph.com/organizations/ceph/?query=70296c9f8938423f9cb53f04cabee65c Command failed on smithi086 with status 5: 'sudo systemctl stop ceph-0ba6cb96-de2c-11ee-95c7-87774f69a715@mon.a' [7587243] rados/cephadm/workunits/{0-distro/centos_9.stream agent/on mon_election/connectivity task/test_host_drain} ----------------------------------------------------------------- time: 00:08:25 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587243/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587243/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e48bde6c6cae4958843eebe56e3f1763 Command failed on smithi052 with status 5: 'sudo systemctl stop ceph-67092596-de2d-11ee-95c7-87774f69a715@mon.a' [7587251] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{centos_latest}} ----------------------------------------------------------------- time: 00:07:26 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587251/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587251/ sentry: https://sentry.ceph.com/organizations/ceph/?query=13e589b8aacb43e5b340a86ac874580d Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_lazy_omap_stats' [7587267] rados/cephadm/smoke/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:05:43 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587267/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587267/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d916d17903b44a1fa943e5b7398e2c61 Command failed on smithi143 with status 5: 'sudo systemctl stop ceph- aef95b8e-de35-11ee-95c7-87774f69a715@mon.a' [7587275] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} ----------------------------------------------------------------- time: 00:20:11 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587275/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587275/ valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun [7587278] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} ----------------------------------------------------------------- time: 00:27:20 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587278/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587278/ valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun [7587299] rados/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_iscsi_container/{centos_9.stream test_iscsi_container}} ----------------------------------------------------------------- time: 00:08:36 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587299/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587299/ sentry: https://sentry.ceph.com/organizations/ceph/?query=37f5d35538b14f7b8a426206529409c7 Command failed on smithi032 with status 5: 'sudo systemctl stop ceph-4014f780-de37-11ee-95c7-87774f69a715@mon.a' [7587302] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/snaps-few-objects} ----------------------------------------------------------------- time: 00:10:40 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587302/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587302/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4fe93dda419848e3867225e3d2bb5409 Command failed on smithi037 with status 5: 'sudo systemctl stop ceph-23f48038-de38-11ee-95c7-87774f69a715@mon.a' [7587335] rados/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/repave-all} ----------------------------------------------------------------- time: 00:05:48 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587335/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587335/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b2df45f466854ca5b96618fdd3a69260 Command failed on smithi018 with status 5: 'sudo systemctl stop ceph-3b2a0ca4-de39-11ee-95c7-87774f69a715@mon.smithi018' [7587368] rados/cephadm/workunits/{0-distro/ubuntu_22.04 agent/on mon_election/connectivity task/test_monitoring_stack_basic} ----------------------------------------------------------------- time: 00:11:21 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587368/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587368/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e754788d220e478295f2d9e193c1e67c Command failed on smithi044 with status 5: 'sudo systemctl stop ceph-30e72fe4-de3c-11ee-95c7-87774f69a715@mon.a' [7587406] rados/cephadm/osds/{0-distro/ubuntu_22.04 0-nvme-loop 1-start 2-ops/rm-zap-add} ----------------------------------------------------------------- time: 00:08:53 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587406/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587406/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b9ab227377fd45be9ea6a487a0ac45ef Command failed on smithi062 with status 5: 'sudo systemctl stop ceph-a0e0e0d2-de3d-11ee-95c7-87774f69a715@mon.smithi062' [7587409] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/test_rbd_api} ----------------------------------------------------------------- time: 00:10:21 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587409/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587409/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1efacd78283e4ec3b8f1d2ae454a0383 Command failed on smithi106 with status 5: 'sudo systemctl stop ceph- de6bbf80-de3d-11ee-95c7-87774f69a715@mon.a' [7587434] rados/upgrade/parallel/{0-random-distro$/{centos_9.stream_runc} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} ----------------------------------------------------------------- time: 00:19:28 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587434/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587434/ sentry: https://sentry.ceph.com/organizations/ceph/?query=37c3c41ebf444cf48329d98df7a312d2 Command failed on smithi029 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7bbfe926-de63-11ee-95c7-87774f69a715 -e sha1=6b01fe95f3aa686f1951a0c5828907a52930b92a -- bash -c 'ceph orch ps'" [7587445] rados/cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_rgw_multisite} ----------------------------------------------------------------- time: 00:08:34 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587445/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587445/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d9439bc99f16469798b7c95da8d77499 Command failed on smithi081 with status 5: 'sudo systemctl stop ceph-3303ce2a-de66-11ee-95c7-87774f69a715@mon.a' [7587472] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} ----------------------------------------------------------------- time: 03:19:58 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587472/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587472/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e8d94e8ebc564171a32df36ab4073b34 Command failed (workunit test cls/test_cls_2pc_queue.sh) on smithi046 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6b01fe95f3aa686f1951a0c5828907a52930b92a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cls/test_cls_2pc_queue.sh' [7587481] rados/cephadm/smoke/{0-distro/ubuntu_22.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:08:54 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587481/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587481/ sentry: https://sentry.ceph.com/organizations/ceph/?query=401b9bd21e3c409d9870d3b2db5553c9 Command failed on smithi138 with status 5: 'sudo systemctl stop ceph-a3736efc-de68-11ee-95c7-87774f69a715@mon.a' [7587508] rados/standalone/{supported-random-distro$/{centos_latest} workloads/mon} ----------------------------------------------------------------- time: 00:12:38 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587508/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587508/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1ebcadcb9bb7460da6395948a35b7ba5 Command failed (workunit test mon/mkfs.sh) on smithi155 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6b01fe95f3aa686f1951a0c5828907a52930b92a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/mon/mkfs.sh' [7587511] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_9.stream_runc} 1-start 2-services/rgw 3-final} ----------------------------------------------------------------- time: 00:05:20 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587511/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587511/ sentry: https://sentry.ceph.com/organizations/ceph/?query=eeb4b21efe9a4ea28038b749c37472a9 Command failed on smithi130 with status 5: 'sudo systemctl stop ceph-171c0962-de6a-11ee-95c7-87774f69a715@mon.smithi130' [7587523] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/cache-snaps} ----------------------------------------------------------------- time: 00:11:20 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587523/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587523/ sentry: https://sentry.ceph.com/organizations/ceph/?query=70be71a79500429a805e18c1bbdce743 Command failed on smithi079 with status 5: 'sudo systemctl stop ceph-650aab50-de6b-11ee-95c7-87774f69a715@mon.a' [7587535] rados/singleton-nomsgr/{all/ceph-snapmapper mon_election/classic rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:10:06 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587535/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587535/ sentry: https://sentry.ceph.com/organizations/ceph/?query=307a0138ed054362a6a658b6e852ce14 Command failed on smithi148 with status 1: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_snap_mapper' [7587552] rados/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_set_mon_crush_locations} ----------------------------------------------------------------- time: 00:08:44 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587552/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587552/ sentry: https://sentry.ceph.com/organizations/ceph/?query=84bab084e12d4b11b3f8ef8d3cf53fd2 Command failed on smithi055 with status 5: 'sudo systemctl stop ceph-666b3008-de6d-11ee-95c7-87774f69a715@mon.a' [7587573] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} ----------------------------------------------------------------- time: 00:20:08 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587573/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587573/ reached maximum tries (51) after waiting for 300 seconds [7587587] rados/cephadm/osds/{0-distro/centos_9.stream 0-nvme-loop 1-start 2-ops/rm-zap-flag} ----------------------------------------------------------------- time: 00:05:28 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587587/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587587/ sentry: https://sentry.ceph.com/organizations/ceph/?query=02d83940ac904a93bdd7c36b90e4e7ec Command failed on smithi019 with status 5: 'sudo systemctl stop ceph-6313ed08-de6f-11ee-95c7-87774f69a715@mon.smithi019' [7587629] rados/cephadm/workunits/{0-distro/centos_9.stream_runc agent/off mon_election/classic task/test_ca_signed_key} ----------------------------------------------------------------- time: 00:08:26 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587629/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587629/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4fdc8e4b4b65430486812e86ce9243c3 Command failed on smithi086 with status 5: 'sudo systemctl stop ceph-4d294288-de72-11ee-95c7-87774f69a715@mon.a' [7587632] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/quincy backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/radosbench} ----------------------------------------------------------------- time: 00:11:45 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587632/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587632/ sentry: https://sentry.ceph.com/organizations/ceph/?query=41f76d81f49a4cb085570d9b9aa0c92e Command failed on smithi142 with status 5: 'sudo systemctl stop ceph-22abe38e-de73-11ee-95c7-87774f69a715@mon.a' [7587650] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-low-osd-mem-target} tasks/e2e} ----------------------------------------------------------------- time: 00:08:08 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587650/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587650/ sentry: https://sentry.ceph.com/organizations/ceph/?query=35157409b471446099087a8512515f4a Command failed on smithi112 with status 5: 'sudo systemctl stop ceph-e8c81650-de73-11ee-95c7-87774f69a715@mon.a' [7587665] rados/cephadm/osds/{0-distro/centos_9.stream_runc 0-nvme-loop 1-start 2-ops/rm-zap-wait} ----------------------------------------------------------------- time: 00:06:07 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587665/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587665/ sentry: https://sentry.ceph.com/organizations/ceph/?query=da1f41262dd84da0bffe539d4a14ab00 Command failed on smithi029 with status 5: 'sudo systemctl stop ceph-722415ba-de75-11ee-95c7-87774f69a715@mon.smithi029' [7587674] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} ----------------------------------------------------------------- time: 00:19:17 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587674/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587674/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0dfae600dbbe4b99bc74340210fe738e valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun [7587680] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:10:04 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587680/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587680/ sentry: https://sentry.ceph.com/organizations/ceph/?query=38b9d623a88d46f690d41132ee094ca7 Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_lazy_omap_stats' [7587698] rados/cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:05:15 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587698/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587698/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c4d5e7906dc54708aca95f24f68b822d Command failed on smithi069 with status 5: 'sudo systemctl stop ceph-53e2d85a-de77-11ee-95c7-87774f69a715@mon.a' Dead ================================================================= [7587059] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587059/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587059/ Error reimaging machines: Failed to power on smithi158 [7587066] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587066/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587066/ Error reimaging machines: Failed to power on smithi158 [7587081] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587081/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587081/ hit max job timeout [7587183] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{centos_latest}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587183/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587183/ Error reimaging machines: reached maximum tries (101) after waiting for 600 seconds [7587281] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587281/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587281/ Error reimaging machines: Failed to power on smithi122 [7587377] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{centos_latest}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587377/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587377/ Error reimaging machines: Failed to power on smithi158 [7587386] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/rados_api_tests} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587386/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587386/ Error reimaging machines: reached maximum tries (101) after waiting for 600 seconds [7587428] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{centos_latest}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587428/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587428/ hit max job timeout Pass ================================================================= [7586984] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} tasks/mon_recovery} time: 00:11:32 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7586984/ [7586986] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:27:44 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7586986/ [7586988] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:14:04 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7586988/ [7586990] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/dedup-io-mixed} time: 00:13:20 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7586990/ [7586992] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest}} time: 00:09:15 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7586992/ [7586994] rados/objectstore/{backends/objectstore-memstore supported-random-distro$/{centos_latest}} time: 00:07:20 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7586994/ [7586996] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects} time: 00:22:04 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7586996/ [7587002] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:17:58 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587002/ [7587004] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/dedup-io-snaps} time: 00:15:01 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587004/ [7587006] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:23:21 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587006/ [7587008] rados/singleton/{all/ec-inconsistent-hinfo mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest}} time: 00:15:22 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587008/ [7587010] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} tasks/libcephsqlite} time: 00:12:11 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587010/ [7587014] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:13:15 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587014/ [7587015] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/pool-snaps-few-objects} time: 00:25:10 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587015/ [7587017] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:20:18 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587017/ [7587019] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:14:03 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587019/ [7587025] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-hybrid} supported-random-distro$/{centos_latest} tasks/workunits} time: 00:08:51 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587025/ [7587028] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}} time: 00:38:42 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587028/ [7587030] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:43:03 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587030/ [7587033] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:10:29 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587033/ [7587035] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_mon_workunits} time: 00:31:53 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587035/ [7587037] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}} time: 00:07:13 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587037/ [7587039] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/radosbench-high-concurrency} time: 00:34:45 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587039/ [7587041] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:10:13 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587041/ [7587043] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:28:19 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587043/ [7587047] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:15:02 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587047/ [7587049] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest}} time: 00:28:50 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587049/ [7587051] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/radosbench} time: 01:20:29 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587051/ [7587053] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:13:07 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587053/ [7587055] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/rados_api_tests} time: 00:44:54 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587055/ [7587057] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:40:32 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587057/ [7587061] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}} time: 00:30:46 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587061/ [7587063] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/redirect} time: 00:13:19 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587063/ [7587068] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest}} time: 00:26:00 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587068/ [7587073] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-stupid} tasks/dashboard} time: 01:11:57 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587073/ [7587075] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:12:37 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587075/ [7587077] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{centos_latest}} time: 00:35:54 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587077/ [7587079] rados/rest/{mgr-restful supported-random-distro$/{ubuntu_latest}} time: 00:12:22 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587079/ [7587083] rados/standalone/{supported-random-distro$/{centos_latest} workloads/c2c} time: 00:13:52 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587083/ [7587087] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:22:28 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587087/ [7587089] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados tasks/mon_recovery validater/lockdep} time: 00:11:53 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587089/ [7587091] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/redirect_promote_tests} time: 00:16:09 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587091/ [7587093] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:30 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587093/ [7587095] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:10:09 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587095/ [7587097] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/redirect_set_object} time: 00:13:29 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587097/ [7587099] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:11:35 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587099/ [7587103] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:13:59 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587103/ [7587105] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/snaps-few-objects} time: 00:36:07 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587105/ [7587107] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:14:11 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587107/ [7587109] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} tasks/rados_cls_all} time: 00:15:41 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587109/ [7587111] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/set-chunks-read} time: 00:16:51 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587111/ [7587113] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} tasks/mon_clock_with_skews} time: 00:07:48 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587113/ [7587115] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:13:23 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587115/ [7587117] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:11:30 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587117/ [7587119] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:21:03 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587119/ [7587121] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest}} time: 00:12:05 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587121/ [7587123] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/small-objects-balanced} time: 00:20:57 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587123/ [7587126] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:10:54 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587126/ [7587128] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:23:51 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587128/ [7587130] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/small-objects-localized} time: 00:19:17 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587130/ [7587132] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:12:34 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587132/ [7587134] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:18 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587134/ [7587136] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{ubuntu_latest} tasks/failover} time: 00:15:18 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587136/ [7587138] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/crush} time: 00:23:50 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587138/ [7587140] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:13:25 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587140/ [7587141] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:07:16 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587141/ [7587143] rados/cephadm/workunits/{0-distro/centos_9.stream_runc agent/on mon_election/connectivity task/test_cephadm_repos} time: 00:05:51 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587143/ [7587145] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/small-objects} time: 00:25:37 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587145/ [7587151] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/rados_python} time: 00:17:16 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587151/ [7587153] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:12:24 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587153/ [7587155] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:41 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587155/ [7587159] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-balanced} time: 00:27:05 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587159/ [7587161] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:16:14 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587161/ [7587163] rados/objectstore/{backends/fusestore supported-random-distro$/{ubuntu_latest}} time: 00:09:45 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587163/ [7587165] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/sync-many workloads/pool-create-delete} time: 00:20:06 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587165/ [7587169] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/snaps-few-objects-localized} time: 00:25:38 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587169/ [7587171] rados/singleton-nomsgr/{all/crushdiff mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:09:03 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587171/ [7587173] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} tasks/mon_recovery} time: 00:22:33 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587173/ [7587175] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:26:12 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587175/ [7587177] rados/singleton/{all/mon-config mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:16:04 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587177/ [7587179] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:14:36 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587179/ [7587181] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/snaps-few-objects} time: 00:26:35 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587181/ [7587185] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_latest} tasks/insights} time: 00:13:55 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587185/ [7587187] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} tasks/rados_stress_watch} time: 00:14:22 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587187/ [7587191] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:24:21 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587191/ [7587193] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:22:03 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587193/ [7587195] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:22:24 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587195/ [7587197] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/write_fadvise_dontneed} time: 00:13:16 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587197/ [7587199] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/erasure-code} time: 01:32:52 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587199/ [7587201] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:23:39 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587201/ [7587203] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:08:09 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587203/ [7587205] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:33:42 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587205/ [7587207] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/admin_socket_objecter_requests} time: 00:23:22 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587207/ [7587209] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:22:41 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587209/ [7587211] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep} time: 00:17:49 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587211/ [7587213] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:14:19 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587213/ [7587219] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:12:11 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587219/ [7587221] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:14:17 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587221/ [7587223] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/cache-agent-big} time: 00:14:22 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587223/ [7587225] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest}} time: 00:11:47 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587225/ [7587229] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} thrashers/sync workloads/rados_5925} time: 00:09:52 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587229/ [7587231] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/cache-agent-small} time: 00:18:09 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587231/ [7587233] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:08:40 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587233/ [7587235] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/rados_striper} time: 00:11:32 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587235/ [7587237] rados/singleton/{all/peer mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest}} time: 00:08:26 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587237/ [7587239] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:09:53 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587239/ [7587241] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:24:52 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587241/ [7587245] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{centos_latest} tasks/module_selftest} time: 00:23:42 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587245/ [7587247] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:18:31 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587247/ [7587249] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{ubuntu_latest}} time: 00:16:20 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587249/ [7587253] rados/standalone/{supported-random-distro$/{centos_latest} workloads/mgr} time: 00:14:52 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587253/ [7587255] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:18:54 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587255/ [7587257] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-radosbench} time: 01:48:26 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587257/ [7587259] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps} time: 00:20:57 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587259/ [7587261] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:12:13 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587261/ [7587263] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:17 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587263/ [7587265] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:10:30 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587265/ [7587269] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}} time: 00:15:35 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587269/ [7587272] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/cache-snaps-balanced} time: 00:20:21 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587272/ [7587284] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:18 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587284/ [7587287] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:20:23 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587287/ [7587290] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} tasks/rados_workunit_loadgen_big} time: 00:29:41 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587290/ [7587293] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/cache-snaps} time: 00:25:20 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587293/ [7587296] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}} time: 00:08:10 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587296/ [7587305] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:21:08 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587305/ [7587308] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} thrashers/force-sync-many workloads/rados_api_tests} time: 00:42:57 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587308/ [7587311] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/cache} time: 00:14:22 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587311/ [7587314] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_latest} tasks/per_module_finisher_stats} time: 00:11:30 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587314/ [7587317] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:22:57 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587317/ [7587320] rados/singleton/{all/radostool mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:20:01 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587320/ [7587323] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:09:52 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587323/ [7587326] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:13:13 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587326/ [7587329] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/dedup-io-mixed} time: 00:11:24 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587329/ [7587332] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:20:15 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587332/ [7587338] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/misc} time: 01:07:37 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587338/ [7587341] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:28:40 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587341/ [7587344] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/dedup-io-snaps} time: 00:11:11 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587344/ [7587347] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-small-objects-balanced} time: 00:21:59 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587347/ [7587350] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:22 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587350/ [7587353] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:08:25 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587353/ [7587356] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} tasks/rados_workunit_loadgen_mix} time: 00:18:59 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587356/ [7587359] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{ubuntu_latest}} time: 00:37:54 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587359/ [7587362] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest}} time: 00:12:42 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587362/ [7587365] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/pool-snaps-few-objects} time: 00:28:26 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587365/ [7587371] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} time: 00:25:19 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587371/ [7587374] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:14:00 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587374/ [7587380] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:14:14 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587380/ [7587383] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest}} time: 00:14:07 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587383/ [7587389] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}} time: 00:30:57 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587389/ [7587392] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{centos_latest} tasks/progress} time: 00:19:10 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587392/ [7587395] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:09:19 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587395/ [7587398] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:19:45 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587398/ [7587401] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/radosbench-high-concurrency} time: 01:07:16 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587401/ [7587403] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_mon_osdmap_prune} time: 00:57:36 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587403/ [7587412] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest}} time: 00:08:50 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587412/ [7587416] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} tasks/mon_recovery} time: 00:16:53 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587416/ [7587419] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:26:36 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587419/ [7587422] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} tasks/rados_workunit_loadgen_mostlyread} time: 00:24:43 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587422/ [7587425] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-hybrid} tasks/dashboard} time: 01:09:52 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587425/ [7587431] rados/standalone/{supported-random-distro$/{centos_latest} workloads/mon-stretch} time: 00:22:22 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587431/ [7587437] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/radosbench} time: 00:51:47 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587437/ [7587440] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest}} time: 00:10:22 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587440/ [7587443] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:12:15 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587443/ [7587449] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/redirect} time: 00:20:03 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587449/ [7587451] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:11:33 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587451/ [7587454] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-fast-read} time: 00:21:36 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587454/ [7587457] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:24:00 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587457/ [7587460] rados/singleton/{all/test-noautoscale-flag mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}} time: 00:09:22 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587460/ [7587463] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/redirect_promote_tests} time: 00:10:46 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587463/ [7587466] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:11:22 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587466/ [7587469] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:02 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587469/ [7587475] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-stupid} supported-random-distro$/{centos_latest} tasks/prometheus} time: 00:10:39 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587475/ [7587478] rados/singleton/{all/thrash-backfill-full mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}} time: 02:14:41 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587478/ [7587484] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/redirect_set_object} time: 00:14:46 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587484/ [7587487] rados/objectstore/{backends/objectstore-bluestore-a supported-random-distro$/{centos_latest}} time: 02:14:36 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587487/ [7587490] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} tasks/readwrite} time: 00:11:28 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587490/ [7587493] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:10:54 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587493/ [7587496] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:14:32 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587496/ [7587499] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/set-chunks-read} time: 00:16:39 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587499/ [7587502] rados/singleton/{all/thrash-eio mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:42:33 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587502/ [7587505] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/rados_mon_workunits} time: 00:30:01 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587505/ [7587514] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:09:34 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587514/ [7587517] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:10:22 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587517/ [7587520] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:23:42 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587520/ [7587526] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/small-objects-balanced} time: 00:22:34 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587526/ [7587529] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:21:53 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587529/ [7587532] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites} time: 00:19:31 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587532/ [7587538] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/small-objects-localized} time: 00:24:12 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587538/ [7587541] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:23:39 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587541/ [7587545] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:43:56 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587545/ [7587548] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest}} time: 00:18:59 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587548/ [7587554] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} tasks/repair_test} time: 00:15:42 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587554/ [7587557] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/small-objects} time: 00:21:59 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587557/ [7587560] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:14:09 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587560/ [7587564] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-hybrid} supported-random-distro$/{centos_latest} tasks/workunits} time: 00:08:59 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587564/ [7587567] rados/singleton-nomsgr/{all/crushdiff mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:08:34 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587567/ [7587570] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:11:44 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587570/ [7587576] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} time: 00:12:21 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587576/ [7587578] rados/singleton/{all/watch-notify-same-primary mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest}} time: 00:08:01 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587578/ [7587582] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/snaps-few-objects-balanced} time: 00:25:41 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587582/ [7587585] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:10:09 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587585/ [7587591] rados/singleton/{all/admin-socket mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest}} time: 00:07:47 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587591/ [7587594] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/snaps-few-objects-localized} time: 00:28:17 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587594/ [7587596] rados/standalone/{supported-random-distro$/{centos_latest} workloads/osd-backfill} time: 02:26:20 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587596/ [7587599] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}} time: 00:30:54 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587599/ [7587602] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/sync-many workloads/pool-create-delete} time: 00:16:13 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587602/ [7587605] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:10:39 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587605/ [7587608] rados/objectstore/{backends/objectstore-bluestore-b supported-random-distro$/{centos_latest}} time: 00:09:22 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587608/ [7587611] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} tasks/mon_clock_with_skews} time: 00:07:22 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587611/ [7587614] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:13:16 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587614/ [7587617] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/snaps-few-objects} time: 00:31:17 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587617/ [7587620] rados/singleton/{all/backfill-toofull mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:21:36 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587620/ [7587623] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:13:03 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587623/ [7587626] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/scrub_test} time: 00:13:34 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587626/ [7587635] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:12:19 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587635/ [7587638] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/write_fadvise_dontneed} time: 00:16:00 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587638/ [7587641] rados/singleton/{all/deduptool mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}} time: 00:10:27 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587641/ [7587644] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:35 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587644/ [7587647] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects} time: 00:22:18 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587647/ [7587653] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:12:50 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587653/ [7587656] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:42:45 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587656/ [7587659] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/admin_socket_objecter_requests} time: 00:18:38 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587659/ [7587662] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:10:54 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587662/ [7587668] rados/singleton/{all/divergent_priors mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:12:37 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587668/ [7587671] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:14:20 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587671/ [7587677] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/cache-agent-big} time: 00:47:33 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587677/ [7587683] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:13:21 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587683/ [7587686] rados/singleton/{all/divergent_priors2 mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest}} time: 00:09:16 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587686/ [7587689] rados/standalone/{supported-random-distro$/{centos_latest} workloads/osd} time: 02:30:19 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587689/ [7587692] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite} time: 00:15:36 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587692/ [7587695] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/cache-agent-small} time: 00:11:05 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587695/ [7587701] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} thrashers/sync workloads/rados_5925} time: 00:10:30 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587701/ [7587704] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:12:14 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587704/ [7587707] rados/singleton/{all/dump-stuck mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:11:57 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587707/ [7587710] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:22:08 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587710/ [7587713] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} tasks/mon_recovery} time: 00:11:48 info: https://pulpito.ceph.com/yuriw-2024-03-08_16:18:36-rados-wip-yuri8-testing-2024-03-06-1329-distro-default-smithi/7587713/