2023-06-27T20:18:18.373 INFO:root:teuthology version: 0.0.1.dev130+g3fdf1f2 2023-06-27T20:18:18.374 INFO:teuthology.results:subset: '111/120000' 2023-06-27T20:18:18.375 INFO:teuthology.results:seed: '2812' 2023-06-28T00:04:48.389 INFO:root:teuthology version: 0.0.1.dev132+gaa9345a 2023-06-28T00:04:48.411 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-06-28T02:44:09.119 INFO:teuthology.results:Tests finished! gathering results... 2023-06-28T02:44:41.091 INFO:scrape:Found 307 jobs 2023-06-28T02:44:41.104 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318150/teuthology.log 2023-06-28T02:44:42.716 INFO:scrape:Found 5 distinct failure reasons 2023-06-28T02:44:42.717 INFO:scrape:Dead: None 2023-06-28T02:44:42.717 INFO:scrape:31 jobs 2023-06-28T02:44:42.717 INFO:scrape: 2023-06-28T02:44:42.718 INFO:scrape:Failure: Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi148 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b75c542210c04254e2c94e13c158a5b74292e5f0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh' 2023-06-28T02:44:42.718 INFO:scrape:2 jobs: ['7318311', '7318358'] 2023-06-28T02:44:42.718 INFO:scrape:suites intersection: ['mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v2only', 'rados', 'supported-random-distro$/{rhel_8}}', 'test_envlibrados_for_rocksdb}'] 2023-06-28T02:44:42.719 INFO:scrape:suites union: ['mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v2only', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-low-osd-mem-target', 'rados', 'rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/centos_latest', 'rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/rhel_latest', 'supported-random-distro$/{rhel_8}}', 'test_envlibrados_for_rocksdb}'] 2023-06-28T02:44:42.719 INFO:scrape: 2023-06-28T02:44:42.719 INFO:scrape:Failure: Command failed on smithi195 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic' 2023-06-28T02:44:42.720 INFO:scrape:15 jobs: ['7318420', '7318374', '7318416', '7318406', '7318362', '7318366', '7318122', '7318378', '7318349', '7318341', '7318346', '7318337', '7318384', '7318345', '7318396'] 2023-06-28T02:44:42.720 INFO:scrape:suites intersection: [] 2023-06-28T02:44:42.720 INFO:scrape:suites union: ['0-nvme-loop', '1-pg-log-overrides/short_pg_log', '1-start', '2-ops/rm-zap-add}', '2-recovery-overrides/{default}', '3-scrub-overrides/{max-simultaneous-scrubs-2}', 'agent/off', 'agent/on', 'backoff/peering_and_degraded', 'ceph', 'clusters/{fixed-2', 'crc-failures/default', 'd-balancer/on', 'fast/fast', 'fast/normal', 'fixed-2', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/many', 'msgr-failures/osd-delay', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'no_pools', 'objectstore/bluestore-basic-min-osd-mem-target', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-low-osd-mem-target', 'objectstore/bluestore-stupid', 'openstack', 'openstack}', 'rados', 'rados/basic/{ceph', 'rados/cephadm/osds/{0-distro/ubuntu_20.04', 'rados/cephadm/smoke/{0-distro/ubuntu_20.04', 'rados/cephadm/workunits/{0-distro/ubuntu_20.04', 'rados/multimon/{clusters/21', 'rados/perf/{ceph', 'rados/singleton-bluestore/{all/cephtool', 'rados/singleton-nomsgr/{all/crushdiff', 'rados/singleton/{all/thrash_cache_writeback_proxy_none', 'rados/thrash-erasure-code-isa/{arch/x86_64', 'rados/thrash-erasure-code-overwrites/{bluestore-bitmap', 'rados/thrash-erasure-code/{ceph', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'recovery-overrides/{default}', 'recovery-overrides/{more-async-partial-recovery}', 'recovery-overrides/{more-async-recovery}', 'scheduler/dmclock_1Shard_16Threads', 'scheduler/dmclock_default_shards', 'settings/optimized', 'start}', 'supported-random-distro$/{ubuntu_latest}', 'supported-random-distro$/{ubuntu_latest}}', 'task/test_cephadm_repos}', 'tasks/libcephsqlite}', 'tasks/mon_clock_no_skews}', 'tasks/repair_test}', 'thrashers/mapgap', 'thrashers/minsize_recovery', 'thrashers/pggrow', 'thrashosds-health', 'ubuntu_latest', 'workloads/ec-rados-plugin=isa-k=2-m=1}', 'workloads/ec-small-objects-many-deletes}', 'workloads/ec-snaps-few-objects-overwrites}', 'workloads/fio_4M_rand_read}', 'workloads/fio_4M_rand_rw}', 'workloads/radosbench}'] 2023-06-28T02:44:42.721 INFO:scrape: 2023-06-28T02:44:42.721 INFO:scrape:Failure: Command failed on smithi141 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b75c542210c04254e2c94e13c158a5b74292e5f0 pull' 2023-06-28T02:44:42.721 INFO:scrape:1 jobs: ['7318198'] 2023-06-28T02:44:42.722 INFO:scrape:suites: ['0-nvme-loop', '1-start', '2-ops/rmdir-reactivate}', 'rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0'] 2023-06-28T02:44:42.722 INFO:scrape: 2023-06-28T02:44:42.722 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi101 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b75c542210c04254e2c94e13c158a5b74292e5f0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2023-06-28T02:44:42.723 INFO:scrape:2 jobs: ['7318388', '7318227'] 2023-06-28T02:44:42.723 INFO:scrape:suites intersection: ['debug/mgr', 'rados/dashboard/{0-single-container-host', 'tasks/e2e}'] 2023-06-28T02:44:42.723 INFO:scrape:suites union: ['debug/mgr', 'mon_election/classic', 'mon_election/connectivity', 'rados/dashboard/{0-single-container-host', 'random-objectstore$/{bluestore-comp-snappy}', 'random-objectstore$/{bluestore-hybrid}', 'tasks/e2e}'] 2023-06-28T02:44:42.723 INFO:scrape: 2023-06-28T02:44:42.736 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/ failed: 20 dead: 31 running: 0 waiting: 0 queued: 0 passed: 256 Fail ================================================================= [7318122] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318122/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318122/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f9dfd7369c7a4475a31d0a6f8264cbd6 Command failed on smithi089 with status 1: 'sudo fuser -v /var/lib/dpkg/lock-frontend' [7318198] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} ----------------------------------------------------------------- time: 00:12:17 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318198/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318198/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5767e50bc25b45a89e3d7d99db577672 Command failed on smithi141 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay- quay.apps.os.sepia.ceph.com/ceph- ci/ceph:b75c542210c04254e2c94e13c158a5b74292e5f0 pull' [7318227] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-hybrid} tasks/e2e} ----------------------------------------------------------------- time: 00:27:57 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318227/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318227/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8fc335eedf164f6fb7ce781ed98a41ef Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi006 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b75c542210c04254e2c94e13c158a5b74292e5f0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [7318311] rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/centos_latest test_envlibrados_for_rocksdb} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:14:36 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318311/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318311/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2b2b28665de24af49815deef401581a4 Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi148 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b75c542210c04254e2c94e13c158a5b74292e5f0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh' [7318337] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318337/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318337/ sentry: https://sentry.ceph.com/organizations/ceph/?query=54c32b7d2ece470f83c7e472cbbe7773 Command failed on smithi196 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic' [7318341] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318341/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318341/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9e5875674ee648cdbb1350da9635b7d6 Command failed on smithi156 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic' [7318345] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-many-deletes} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318345/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318345/ sentry: https://sentry.ceph.com/organizations/ceph/?query=573be3948592490293e9be2a18088f67 Command failed on smithi153 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic' [7318346] rados/singleton-nomsgr/{all/crushdiff mon_election/classic rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318346/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318346/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8e5811fe6905455e8f6842273d4831cb Command failed on smithi130 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic' [7318349] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/repair_test} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318349/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318349/ sentry: https://sentry.ceph.com/organizations/ceph/?query=46689163d0254789b4c547791823c417 Command failed on smithi043 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic' [7318358] rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/rhel_latest test_envlibrados_for_rocksdb} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:19:08 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318358/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318358/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2d9e416aabce4c5c9a38ce46f5d22e8e Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi159 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b75c542210c04254e2c94e13c158a5b74292e5f0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh' [7318362] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318362/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318362/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e75d28e8d0b94be0b99b6e2dd513b913 Command failed on smithi134 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic- hwe-20.04' [7318366] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318366/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318366/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c48d5d47c9014df6b2b866aab8fe148f Command failed on smithi032 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic' [7318374] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318374/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318374/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8222cd2cafce4d749d84e83c653cf47a Command failed on smithi202 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic' [7318378] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_cephadm_repos} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318378/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318378/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e5f9ef3ca804442cbde428f8dbefcefa Command failed on smithi110 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic- hwe-20.04' [7318384] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318384/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318384/ sentry: https://sentry.ceph.com/organizations/ceph/?query=80216dde7d8d49e88339da78da7396fa Command failed on smithi078 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic' [7318388] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-snappy} tasks/e2e} ----------------------------------------------------------------- time: 00:27:01 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318388/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318388/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e0279de9047e4bcf88737e3a4a666bff Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi101 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b75c542210c04254e2c94e13c158a5b74292e5f0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [7318396] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318396/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318396/ sentry: https://sentry.ceph.com/organizations/ceph/?query=693dbc835f784249b9455bf405d63b23 Command failed on smithi078 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic' [7318406] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/radosbench} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318406/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318406/ sentry: https://sentry.ceph.com/organizations/ceph/?query=861ab79dc70f4703a43b5262e346e073 Command failed on smithi100 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic' [7318416] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318416/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318416/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ef7de73b9efd42b3a930f77286c7d000 Command failed on smithi062 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic' [7318420] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318420/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318420/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0e93357b2daf4bf58d0e6e8a4d3a3b7f Command failed on smithi195 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic' Dead ================================================================= [7318095] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-snaps-balanced} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318095/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318095/ Error reimaging machines: [Errno 104] Connection reset by peer [7318098] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/snaps-few-objects} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318098/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318098/ Error reimaging machines: [Errno 104] Connection reset by peer [7318099] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_cephadm_repos} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318099/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318099/ Error reimaging machines: [Errno 104] Connection reset by peer [7318127] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/many workloads/rados_mon_workunits} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318127/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318127/ Error reimaging machines: [Errno 104] Connection reset by peer [7318148] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-stupid} tasks/dashboard} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318148/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318148/ Error reimaging machines: [Errno 104] Connection reset by peer [7318150] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318150/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318150/ Error reimaging machines: [Errno 104] Connection reset by peer [7318155] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318155/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318155/ Error reimaging machines: [Errno 104] Connection reset by peer [7318156] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados tasks/mon_recovery validater/lockdep} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318156/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318156/ Error reimaging machines: [Errno 104] Connection reset by peer [7318169] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/radosbench} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318169/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318169/ Error reimaging machines: [Errno 104] Connection reset by peer [7318175] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318175/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318175/ Error reimaging machines: [Errno 104] Connection reset by peer [7318178] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318178/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318178/ Error reimaging machines: [Errno 104] Connection reset by peer [7318182] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318182/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318182/ Error reimaging machines: [Errno 104] Connection reset by peer [7318185] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318185/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318185/ Error reimaging machines: [Errno 104] Connection reset by peer [7318199] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} tasks/mon_recovery} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318199/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318199/ Error reimaging machines: [Errno 104] Connection reset by peer [7318200] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318200/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318200/ Error reimaging machines: [Errno 104] Connection reset by peer [7318202] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/set-chunks-read} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318202/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318202/ Error reimaging machines: [Errno 104] Connection reset by peer [7318203] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318203/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318203/ Error reimaging machines: [Errno 104] Connection reset by peer [7318229] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/radosbench} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318229/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318229/ Error reimaging machines: [Errno 104] Connection reset by peer [7318239] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{ubuntu_20.04}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318239/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318239/ Error reimaging machines: [Errno 104] Connection reset by peer [7318240] rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0} 1-start 2-services/rgw 3-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318240/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318240/ Error reimaging machines: [Errno 104] Connection reset by peer [7318245] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318245/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318245/ Error reimaging machines: [Errno 104] Connection reset by peer [7318253] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318253/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318253/ Error reimaging machines: [Errno 104] Connection reset by peer [7318256] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318256/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318256/ Error reimaging machines: [Errno 104] Connection reset by peer [7318268] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318268/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318268/ Error reimaging machines: [Errno 104] Connection reset by peer [7318269] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318269/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318269/ Error reimaging machines: [Errno 104] Connection reset by peer [7318320] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rmdir-reactivate} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318320/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318320/ Error reimaging machines: [Errno 104] Connection reset by peer [7318350] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318350/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318350/ Error reimaging machines: [Errno 104] Connection reset by peer [7318351] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318351/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318351/ Error reimaging machines: [Errno 104] Connection reset by peer [7318354] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318354/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318354/ Error reimaging machines: [Errno 104] Connection reset by peer [7318359] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318359/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318359/ Error reimaging machines: [Errno 104] Connection reset by peer [7318410] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318410/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318410/ Error reimaging machines: [Errno 104] Connection reset by peer Pass ================================================================= [7318094] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/rados_mon_osdmap_prune} time: 00:44:47 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318094/ [7318096] rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:12:53 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318096/ [7318097] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:13:41 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318097/ [7318100] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:15:28 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318100/ [7318101] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:21:09 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318101/ [7318102] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:14:32 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318102/ [7318103] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} time: 00:18:48 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318103/ [7318104] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:26:38 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318104/ [7318105] rados/objectstore/{backends/objectstore-memstore supported-random-distro$/{rhel_8}} time: 00:26:01 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318105/ [7318106] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:21:08 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318106/ [7318107] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-small-objects} time: 00:29:51 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318107/ [7318108] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:22:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318108/ [7318109] rados/singleton/{all/ec-inconsistent-hinfo mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04}} time: 00:17:16 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318109/ [7318110] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:32 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318110/ [7318111] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache} time: 00:24:06 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318111/ [7318112] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:22:34 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318112/ [7318113] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} tasks/libcephsqlite} time: 00:15:33 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318113/ [7318114] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:18:15 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318114/ [7318115] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:29:25 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318115/ [7318116] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados tasks/rados_cls_all validater/valgrind} time: 02:04:19 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318116/ [7318117] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:13:51 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318117/ [7318118] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:36:30 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318118/ [7318119] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:24:50 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318119/ [7318120] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/scrub} time: 02:47:21 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318120/ [7318121] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-bitmap} supported-random-distro$/{rhel_8} tasks/workunits} time: 00:16:50 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318121/ [7318123] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/dedup-io-mixed} time: 00:12:25 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318123/ [7318124] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:32:06 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318124/ [7318125] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:09:22 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318125/ [7318126] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04}} time: 00:08:25 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318126/ [7318128] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_nfs} time: 00:32:29 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318128/ [7318129] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/dedup-io-snaps} time: 00:18:54 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318129/ [7318130] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} tasks/mon_clock_no_skews} time: 00:09:04 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318130/ [7318131] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:27:18 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318131/ [7318132] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:29:03 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318132/ [7318133] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:24:05 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318133/ [7318134] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:14:01 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318134/ [7318135] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/rados_api_tests} time: 00:33:46 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318135/ [7318136] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/pool-snaps-few-objects} time: 00:29:24 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318136/ [7318137] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_orch_cli} time: 00:19:37 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318137/ [7318138] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/test_rbd_api} time: 00:22:33 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318138/ [7318139] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04}} time: 00:32:14 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318139/ [7318140] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:17:28 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318140/ [7318141] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:31:52 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318141/ [7318142] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:46:19 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318142/ [7318143] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:26:21 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318143/ [7318144] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:14:19 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318144/ [7318145] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:31:31 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318145/ [7318146] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/rados_api_tests} time: 00:34:22 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318146/ [7318147] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:14:33 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318147/ [7318149] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{centos_8} tasks/crash} time: 00:19:14 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318149/ [7318151] rados/rest/{mgr-restful supported-random-distro$/{ubuntu_20.04}} time: 00:15:25 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318151/ [7318152] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:22:01 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318152/ [7318153] rados/standalone/{supported-random-distro$/{centos_8} workloads/c2c} time: 00:17:04 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318153/ [7318154] rados/upgrade/parallel/{0-random-distro$/{centos_8.stream_container_tools} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:29:24 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318154/ [7318157] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_orch_cli_mon} time: 00:30:32 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318157/ [7318158] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:14:35 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318158/ [7318159] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:14:38 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318159/ [7318160] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/radosbench-high-concurrency} time: 00:47:59 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318160/ [7318161] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:16:10 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318161/ [7318162] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:15:23 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318162/ [7318163] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:14:41 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318163/ [7318164] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/one workloads/snaps-few-objects} time: 00:39:54 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318164/ [7318165] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/rados_cls_all} time: 00:19:16 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318165/ [7318166] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:16:23 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318166/ [7318167] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/basic 3-final} time: 00:12:38 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318167/ [7318168] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:10:37 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318168/ [7318170] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:32:52 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318170/ [7318171] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:17:10 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318171/ [7318172] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:14:06 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318172/ [7318173] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:17:13 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318173/ [7318174] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/redirect} time: 00:12:54 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318174/ [7318176] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:32:49 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318176/ [7318177] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_rgw_multisite} time: 00:18:19 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318177/ [7318179] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{ubuntu_latest} tasks/failover} time: 00:15:36 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318179/ [7318180] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04}} time: 00:11:17 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318180/ [7318181] rados/standalone/{supported-random-distro$/{rhel_8} workloads/crush} time: 00:29:51 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318181/ [7318183] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:13:26 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318183/ [7318184] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:14:55 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318184/ [7318186] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/redirect_promote_tests} time: 00:14:36 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318186/ [7318187] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/cache-snaps} time: 00:35:29 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318187/ [7318188] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} time: 00:49:57 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318188/ [7318189] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/rados_python} time: 00:17:02 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318189/ [7318190] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:42 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318190/ [7318191] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:22:30 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318191/ [7318192] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_set_mon_crush_locations} time: 00:22:59 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318192/ [7318193] rados/singleton-nomsgr/{all/crushdiff mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:15:01 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318193/ [7318194] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/redirect_set_object} time: 00:15:19 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318194/ [7318195] rados/objectstore/{backends/fusestore supported-random-distro$/{ubuntu_latest}} time: 00:10:18 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318195/ [7318196] rados/singleton/{all/mon-config mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04}} time: 00:10:35 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318196/ [7318197] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/sync-many workloads/pool-create-delete} time: 00:27:22 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318197/ [7318201] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_adoption} time: 00:15:30 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318201/ [7318204] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:21:18 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318204/ [7318205] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{rhel_8} tasks/insights} time: 00:23:34 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318205/ [7318206] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:19:26 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318206/ [7318207] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} tasks/rados_stress_watch} time: 00:18:34 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318207/ [7318208] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:09:32 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318208/ [7318209] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:30:13 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318209/ [7318210] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:21:36 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318210/ [7318211] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:27:39 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318211/ [7318212] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/small-objects-balanced} time: 00:21:47 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318212/ [7318213] rados/standalone/{supported-random-distro$/{rhel_8} workloads/erasure-code} time: 01:30:18 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318213/ [7318214] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:27 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318214/ [7318215] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_cephadm} time: 00:19:46 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318215/ [7318216] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:38:37 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318216/ [7318217] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:40:16 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318217/ [7318218] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:17:53 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318218/ [7318219] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:13:55 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318219/ [7318220] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep} time: 00:21:19 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318220/ [7318221] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/small-objects-localized} time: 00:27:21 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318221/ [7318222] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:29:10 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318222/ [7318223] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:18:45 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318223/ [7318224] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:14:36 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318224/ [7318225] rados/singleton/{all/peer mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:11:34 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318225/ [7318226] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:22:02 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318226/ [7318228] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:09:39 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318228/ [7318230] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/small-objects} time: 00:21:05 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318230/ [7318231] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/sync workloads/rados_5925} time: 00:18:57 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318231/ [7318232] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_cephadm_repos} time: 00:08:50 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318232/ [7318233] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/rados_striper} time: 00:11:57 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318233/ [7318234] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:16:58 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318234/ [7318235] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:14:46 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318235/ [7318236] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:14:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318236/ [7318237] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{centos_8} tasks/module_selftest} time: 00:32:06 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318237/ [7318238] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/snaps-few-objects-balanced} time: 00:30:33 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318238/ [7318241] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:16:38 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318241/ [7318242] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/mgr} time: 00:15:57 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318242/ [7318243] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:18:46 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318243/ [7318244] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-radosbench} time: 01:21:28 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318244/ [7318246] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:12:51 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318246/ [7318247] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:24:39 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318247/ [7318248] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/snaps-few-objects-localized} time: 00:31:56 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318248/ [7318249] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04}} time: 00:09:15 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318249/ [7318250] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:18:40 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318250/ [7318251] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:18:38 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318251/ [7318252] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:19:50 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318252/ [7318254] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} time: 00:30:52 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318254/ [7318255] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:33:58 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318255/ [7318257] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:27:38 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318257/ [7318258] rados/singleton/{all/radostool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04}} time: 00:15:54 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318258/ [7318259] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_big} time: 00:33:46 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318259/ [7318260] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:24:07 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318260/ [7318261] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:21:13 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318261/ [7318262] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_nfs} time: 00:36:01 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318262/ [7318263] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} thrashers/force-sync-many workloads/rados_api_tests} time: 00:28:29 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318263/ [7318264] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04}} time: 00:22:02 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318264/ [7318265] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/write_fadvise_dontneed} time: 00:24:15 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318265/ [7318266] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{centos_8} tasks/per_module_finisher_stats} time: 00:13:13 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318266/ [7318267] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:24:49 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318267/ [7318270] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:19:06 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318270/ [7318271] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04}} time: 00:14:32 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318271/ [7318272] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/rbd_cls} time: 00:28:23 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318272/ [7318273] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/misc} time: 01:23:19 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318273/ [7318274] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/admin_socket_objecter_requests} time: 00:19:53 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318274/ [7318275] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:15:45 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318275/ [7318276] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:09:40 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318276/ [7318277] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-small-objects-balanced} time: 00:22:02 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318277/ [7318278] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:25:14 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318278/ [7318279] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:35:52 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318279/ [7318280] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mix} time: 00:23:51 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318280/ [7318281] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{ubuntu_20.04}} time: 00:30:22 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318281/ [7318282] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_orch_cli} time: 00:18:48 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318282/ [7318283] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-agent-big} time: 00:20:23 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318283/ [7318284] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:44:30 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318284/ [7318285] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:32:21 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318285/ [7318286] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} time: 00:37:29 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318286/ [7318287] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:11:56 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318287/ [7318288] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:26:49 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318288/ [7318289] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:15:20 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318289/ [7318290] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:14:20 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318290/ [7318291] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache-agent-small} time: 00:20:16 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318291/ [7318292] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04}} time: 00:33:01 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318292/ [7318293] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-stupid} supported-random-distro$/{rhel_8} tasks/progress} time: 00:31:54 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318293/ [7318294] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_orch_cli_mon} time: 00:38:13 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318294/ [7318295] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:13:34 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318295/ [7318296] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:14:31 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318296/ [7318297] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:30:39 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318297/ [7318298] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/many workloads/rados_mon_osdmap_prune} time: 01:07:34 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318298/ [7318299] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:24:35 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318299/ [7318300] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:14:38 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318300/ [7318301] rados/singleton/{all/test-noautoscale-flag mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:11:29 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318301/ [7318302] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:17:10 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318302/ [7318303] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_mostlyread} time: 00:29:59 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318303/ [7318304] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-low-osd-mem-target} tasks/dashboard} time: 01:18:40 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318304/ [7318305] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:30:53 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318305/ [7318306] rados/standalone/{supported-random-distro$/{centos_8} workloads/mon-stretch} time: 00:23:19 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318306/ [7318307] rados/upgrade/parallel/{0-random-distro$/{rhel_8.6_container_tools_3.0} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:30:00 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318307/ [7318308] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_rgw_multisite} time: 00:26:06 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318308/ [7318309] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps} time: 00:28:11 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318309/ [7318310] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:13:24 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318310/ [7318312] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:12:18 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318312/ [7318313] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-small-objects-fast-read} time: 00:29:50 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318313/ [7318314] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:33:56 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318314/ [7318315] rados/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_20.04} 1-start 2-services/basic 3-final} time: 00:17:49 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318315/ [7318316] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/quincy backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} time: 00:43:10 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318316/ [7318317] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:19:21 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318317/ [7318318] rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:59:32 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318318/ [7318319] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/cache-snaps-balanced} time: 00:21:28 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318319/ [7318321] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:50 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318321/ [7318322] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:17:03 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318322/ [7318323] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} time: 02:06:56 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318323/ [7318324] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{ubuntu_latest} tasks/prometheus} time: 00:14:02 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318324/ [7318325] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:33:13 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318325/ [7318326] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:18:38 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318326/ [7318327] rados/objectstore/{backends/objectstore-bluestore-a supported-random-distro$/{ubuntu_latest}} time: 02:50:26 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318327/ [7318328] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache-snaps} time: 00:22:37 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318328/ [7318329] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/readwrite} time: 00:18:23 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318329/ [7318330] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:11:41 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318330/ [7318331] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:15:24 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318331/ [7318332] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_set_mon_crush_locations} time: 00:18:40 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318332/ [7318333] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} thrashers/one workloads/rados_mon_workunits} time: 00:30:47 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318333/ [7318334] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:24:26 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318334/ [7318335] rados/standalone/{supported-random-distro$/{rhel_8} workloads/mon} time: 01:10:26 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318335/ [7318336] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache} time: 00:15:15 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318336/ [7318338] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_adoption} time: 00:24:40 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318338/ [7318339] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:11:48 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318339/ [7318340] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites} time: 00:21:42 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318340/ [7318342] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/repave-all} time: 00:13:33 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318342/ [7318343] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/dedup-io-mixed} time: 00:26:05 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318343/ [7318344] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:31:09 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318344/ [7318347] rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04}} time: 00:09:27 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318347/ [7318348] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:18:04 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318348/ [7318352] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{centos_8} tasks/workunits} time: 00:15:08 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318352/ [7318353] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/dedup-io-snaps} time: 00:17:23 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318353/ [7318355] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:15:57 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318355/ [7318356] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:24:58 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318356/ [7318357] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} time: 00:16:34 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318357/ [7318360] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/test_rbd_api} time: 00:20:49 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318360/ [7318361] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/pool-snaps-few-objects} time: 00:32:53 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318361/ [7318363] rados/singleton/{all/admin-socket mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:18:43 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318363/ [7318364] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:14:54 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318364/ [7318365] rados/standalone/{supported-random-distro$/{rhel_8} workloads/osd-backfill} time: 02:41:33 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318365/ [7318367] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04} thrashers/sync-many workloads/pool-create-delete} time: 00:16:09 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318367/ [7318368] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:20:19 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318368/ [7318369] rados/objectstore/{backends/objectstore-bluestore-b supported-random-distro$/{centos_8}} time: 00:10:25 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318369/ [7318370] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:18:48 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318370/ [7318371] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/rados_api_tests} time: 00:27:48 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318371/ [7318372] rados/singleton/{all/backfill-toofull mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04}} time: 00:21:44 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318372/ [7318373] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:13:22 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318373/ [7318376] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/scrub_test} time: 00:18:21 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318376/ [7318380] rados/singleton/{all/deduptool mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:20:29 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318380/ [7318382] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/radosbench-high-concurrency} time: 00:36:19 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318382/ [7318386] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/ec-small-objects} time: 00:21:40 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318386/ [7318390] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{rhel_8} tasks/crash} time: 00:24:24 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318390/ [7318392] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:16:41 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318392/ [7318394] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:16:31 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318394/ [7318398] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:22:16 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318398/ [7318400] rados/singleton/{all/divergent_priors mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04}} time: 00:10:18 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318400/ [7318402] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:15:06 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318402/ [7318404] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:20:47 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318404/ [7318408] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:19:04 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318408/ [7318412] rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0} 1-start 2-services/rgw 3-final} time: 00:18:44 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318412/ [7318414] rados/singleton/{all/divergent_priors2 mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:23:38 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318414/ [7318418] rados/standalone/{supported-random-distro$/{rhel_8} workloads/osd} time: 02:40:48 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318418/ [7318423] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:13:46 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318423/ [7318425] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/redirect} time: 00:15:27 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318425/ [7318427] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:17:30 info: http://pulpito.ceph.com/yuriw-2023-06-27_20:17:00-rados-wip-yuri11-testing-2023-06-27-0812-reef-distro-default-smithi/7318427/