2023-05-24T18:47:16.367 INFO:root:teuthology version: 0.0.1.dev105+g6fb4f0c
2023-05-24T18:47:16.369 INFO:teuthology.results:subset: '111/120000'
2023-05-24T18:47:16.369 INFO:teuthology.results:seed: '1833'
2023-05-24T19:46:47.791 INFO:root:teuthology version: 0.0.1.dev105+g6fb4f0c
2023-05-24T19:46:47.839 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-05-25T07:35:51.181 INFO:teuthology.results:Tests finished! gathering results...
2023-05-25T07:36:03.846 INFO:scrape:Found 152 jobs
2023-05-25T07:36:05.532 WARNING:scrape:Misunderstood line: 
2023-05-25T07:36:06.111 WARNING:scrape:Misunderstood line: 
2023-05-25T07:36:07.094 WARNING:scrape:Misunderstood line: 
2023-05-25T07:36:07.332 WARNING:scrape:Misunderstood line: 
2023-05-25T07:36:07.358 INFO:scrape:Found 8 distinct failure reasons
2023-05-25T07:36:07.359 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/cls/test_cls_2pc_queue.sh
2023-05-25T07:36:07.360 INFO:scrape:2 jobs: ['7285353', '7285248']
2023-05-25T07:36:07.360 INFO:scrape:suites intersection: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async-v1only', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_cls_all', 'validater/valgrind}']
2023-05-25T07:36:07.360 INFO:scrape:suites union: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async-v1only', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_cls_all', 'validater/valgrind}']
2023-05-25T07:36:07.361 INFO:scrape:
2023-05-25T07:36:07.361 INFO:scrape:Failure: Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri2-testing-2023-05-24-0747
2023-05-25T07:36:07.361 INFO:scrape:6 jobs: ['7285277', '7285267', '7285281', '7285254', '7285304', '7285269']
2023-05-25T07:36:07.361 INFO:scrape:
2023-05-25T07:36:07.362 INFO:scrape:Dead: b'2023-05-25T07:31:12.068 DEBUG:teuthology.exit:Finished running handlers'
2023-05-25T07:36:07.362 INFO:scrape:3 jobs: ['7285342', '7285263', '7285337']
2023-05-25T07:36:07.362 INFO:scrape:suites intersection: []
2023-05-25T07:36:07.363 INFO:scrape:suites union: ['debug/mgr', 'mgr_ttl_cache/enable', 'mon_election/classic', 'mon_election/connectivity', 'rados', 'rados/mgr/{clusters/{2-node-mgr}', 'rados/singleton-nomsgr/{all/admin_socket_output', 'random-objectstore$/{bluestore-low-osd-mem-target}', 'supported-random-distro$/{centos_8}}', 'supported-random-distro$/{centos_latest}', 'tasks/progress}']
2023-05-25T07:36:07.363 INFO:scrape:
2023-05-25T07:36:07.363 INFO:scrape:Failure: Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi124 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2c06d7419cb6b639b0e699e672a277d5e998ef86 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh'
2023-05-25T07:36:07.363 INFO:scrape:2 jobs: ['7285345', '7285370']
2023-05-25T07:36:07.364 INFO:scrape:suites intersection: ['mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v2only', 'rados', 'test_envlibrados_for_rocksdb}']
2023-05-25T07:36:07.364 INFO:scrape:suites union: ['mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v2only', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-low-osd-mem-target', 'rados', 'rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/centos_latest', 'rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/rhel_latest', 'supported-random-distro$/{centos_8}}', 'supported-random-distro$/{centos_latest}}', 'test_envlibrados_for_rocksdb}']
2023-05-25T07:36:07.364 INFO:scrape:
2023-05-25T07:36:07.365 INFO:scrape:Failure: Command failed (workunit test cls/test_cls_rbd.sh) on smithi053 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=quincy TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh'
2023-05-25T07:36:07.365 INFO:scrape:1 jobs: ['7285264']
2023-05-25T07:36:07.365 INFO:scrape:suites: ['0-start', '1-tasks', 'mon_election/classic', 'rados/upgrade/parallel/{0-random-distro$/{centos_8.stream_container_tools}', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'test_rbd_api', 'test_rbd_python}}', 'upgrade-sequence', 'workload/{ec-rados-default']
2023-05-25T07:36:07.366 INFO:scrape:
2023-05-25T07:36:07.366 INFO:scrape:Valgrind: mon (Leak_StillReachable)
2023-05-25T07:36:07.366 INFO:scrape:2 jobs: ['7285315', '7285314']
2023-05-25T07:36:07.366 INFO:scrape:suites intersection: []
2023-05-25T07:36:07.367 INFO:scrape:suites union: ['2-inject-leak/none', 'centos_latest}', 'ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async-v2only', 'objectstore/bluestore-comp-snappy', 'openstack}', 'rados', 'rados/valgrind-leaks/{1-start', 'rados/verify/{centos_latest', 'tasks/mon_recovery', 'validater/valgrind}']
2023-05-25T07:36:07.367 INFO:scrape:
2023-05-25T07:36:07.367 INFO:scrape:Crash: "1684959792.6911714 osd.5 (osd.5) 10 : cluster [WRN] osd.5 PgScrubber: 20.4 timeout on reserving replicsa (since 2023-05-24T20:23:07.684973+0000)" in cluster log
2023-05-25T07:36:07.368 INFO:scrape:ceph version 18.0.0-4138-g2c06d741 (2c06d7419cb6b639b0e699e672a277d5e998ef86) reef (dev)
 1: /lib64/libc.so.6(+0x54df0) [0x549cdf0]
 2: /lib64/libc.so.6(+0xa154c) [0x54e954c]
 3: (ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d const*, char const*, std::chrono::time_point<ceph::coarse_mono_clock, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> > >)+0x226) [0xba34c6]
 4: (ceph::HeartbeatMap::reset_timeout(ceph::heartbeat_handle_d*, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >)+0x70) [0xba35c0]
 5: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x24f) [0xbbae8f]
 6: ceph-osd(+0xab3414) [0xbbb414]
 7: /lib64/libc.so.6(+0x9f802) [0x54e7802]
 8: clone()
2023-05-25T07:36:07.368 INFO:scrape:2 jobs: ['7285387', '7285284']
2023-05-25T07:36:07.368 INFO:scrape:suites intersection: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_api_tests', 'validater/valgrind}']
2023-05-25T07:36:07.368 INFO:scrape:suites union: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_api_tests', 'validater/valgrind}']
2023-05-25T07:36:07.369 INFO:scrape:
2023-05-25T07:36:07.369 INFO:scrape:Failure: Command failed on smithi161 with status 1: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph_test_cls_rbd --gtest_filter=-TestClsRbd.get_features:TestClsRbd.parents:TestClsRbd.mirror'"
2023-05-25T07:36:07.369 INFO:scrape:1 jobs: ['7285326']
2023-05-25T07:36:07.370 INFO:scrape:suites: ['0-size-min-size-overrides/2-size-2-min-size', '1-install/pacific', 'backoff/peering', 'ceph', 'clusters/{openstack', 'd-balancer/crush-compat', 'mon_election/classic', 'msgr-failures/few', 'rados', 'rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools}', 'thrashers/mapgap', 'thrashosds-health', 'three-plus-one}', 'workloads/rbd_cls}']
2023-05-25T07:36:07.370 INFO:scrape:
2023-05-25T07:36:07.375 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/
failed:  16
dead:    3
running: 0
waiting: 0
queued:  0
passed:  133


Fail
=================================================================
[7285248]  rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados tasks/rados_cls_all validater/valgrind}
-----------------------------------------------------------------
time:   03:29:14
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285248/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285248/
sentry: https://sentry.ceph.com/organizations/ceph/?query=72eab27560f8457c99e4b2e19607fae8

    Command failed (workunit test cls/test_cls_2pc_queue.sh) on smithi002 with
    status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=2c06d7419cb6b639b0e699e672a277d5e998ef86
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/cls/test_cls_2pc_queue.sh'

[7285254]  rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/rados_api_tests}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285254/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285254/
sentry: https://sentry.ceph.com/organizations/ceph/?query=3fb374e109f04a519a4303da7a5d48fb

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-
    yuri2-testing-2023-05-24-0747

[7285264]  rados/upgrade/parallel/{0-random-distro$/{centos_8.stream_container_tools} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}}
-----------------------------------------------------------------
time:   00:42:06
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285264/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285264/
sentry: https://sentry.ceph.com/organizations/ceph/?query=70461da586a240ec927da83ca46e8636

    Command failed (workunit test cls/test_cls_rbd.sh) on smithi053 with status
    1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=quincy TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph"
    CEPH_ID="0" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh'

[7285267]  rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_orch_cli_mon}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285267/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285267/
sentry: https://sentry.ceph.com/organizations/ceph/?query=469bde6b2e974f15aa7e980badac7131

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-
    yuri2-testing-2023-05-24-0747

[7285269]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/radosbench-high-concurrency}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285269/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285269/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e7fa551c862a4df3b62bfd8606ac253c

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-
    yuri2-testing-2023-05-24-0747

[7285277]  rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/redirect}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285277/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285277/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4625128ebc254c66938707fdb3e8143c

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-
    yuri2-testing-2023-05-24-0747

[7285281]  rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285281/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285281/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c418cd320f6043e096fd7043005f1d18

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-
    yuri2-testing-2023-05-24-0747

[7285284]  rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind}
-----------------------------------------------------------------
time:   06:26:09
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285284/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285284/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9b6ef58e1c4c42a99f0ac211933e1a06

    Command failed (workunit test rados/test.sh) on smithi031 with status 124:
    'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=2c06d7419cb6b639b0e699e672a277d5e998ef86
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 ALLOW_TIMEOUTS=1 adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 6h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh'

[7285304]  rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_cephadm_repos}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285304/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285304/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9e0522b1d5234215bd3890173e666393

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-
    yuri2-testing-2023-05-24-0747

[7285314]  rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest}
-----------------------------------------------------------------
time:   00:23:48
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285314/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285314/

    saw valgrind issues

[7285315]  rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind}
-----------------------------------------------------------------
time:   00:41:45
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285315/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285315/

    saw valgrind issues

[7285326]  rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/rbd_cls}
-----------------------------------------------------------------
time:   00:23:39
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285326/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285326/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d87603c3275f4c6fa402ff2b53c7ef25

    Command failed on smithi161 with status 1: "sudo
    TESTDIR=/home/ubuntu/cephtest bash -c 'ceph_test_cls_rbd --gtest_filter=-Te
    stClsRbd.get_features:TestClsRbd.parents:TestClsRbd.mirror'"

[7285345]  rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/centos_latest test_envlibrados_for_rocksdb} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest}}
-----------------------------------------------------------------
time:   00:12:02
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285345/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285345/
sentry: https://sentry.ceph.com/organizations/ceph/?query=12c642d8485d4ccdb3b843aac5a25eba

    Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on
    smithi124 with status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=2c06d7419cb6b639b0e699e672a277d5e998ef86
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh'

[7285353]  rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind}
-----------------------------------------------------------------
time:   03:27:21
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285353/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285353/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4168ffba4ac64d24907307f885bafe04

    Command failed (workunit test cls/test_cls_2pc_queue.sh) on smithi149 with
    status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=2c06d7419cb6b639b0e699e672a277d5e998ef86
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/cls/test_cls_2pc_queue.sh'

[7285370]  rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/rhel_latest test_envlibrados_for_rocksdb} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:11:34
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285370/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285370/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0c73f65b3d634ecf9b828dfd4f2f6ea0

    Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on
    smithi154 with status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=2c06d7419cb6b639b0e699e672a277d5e998ef86
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh'

[7285387]  rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind}
-----------------------------------------------------------------
time:   01:06:58
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285387/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285387/

    "1684959792.6911714 osd.5 (osd.5) 10 : cluster [WRN] osd.5 PgScrubber: 20.4
    timeout on reserving replicsa (since 2023-05-24T20:23:07.684973+0000)" in
    cluster log



Dead
=================================================================
[7285263]  rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285263/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285263/

    hit max job timeout

[7285337]  rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{centos_latest} tasks/progress}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285337/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285337/

    hit max job timeout

[7285342]  rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285342/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285342/

    hit max job timeout



Pass
=================================================================
[7285239] rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}}
time:   00:14:55
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285239/

[7285240] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/snaps-few-objects}
time:   00:35:49
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285240/

[7285241] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_cephadm_repos}
time:   00:08:19
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285241/

[7285242] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-snaps}
time:   00:24:25
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285242/

[7285243] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}}
time:   00:21:49
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285243/

[7285244] rados/singleton/{all/ec-inconsistent-hinfo mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}}
time:   00:18:15
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285244/

[7285245] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:28:29
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285245/

[7285246] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}
time:   00:21:29
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285246/

[7285247] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} tasks/libcephsqlite}
time:   00:16:08
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285247/

[7285249] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest}}
time:   00:43:56
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285249/

[7285250] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:20:40
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285250/

[7285251] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{centos_8} tasks/workunits}
time:   00:14:10
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285251/

[7285252] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}}
time:   00:10:56
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285252/

[7285253] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}}
time:   00:26:13
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285253/

[7285255] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/test_rbd_api}
time:   00:26:26
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285255/

[7285256] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{centos_latest}}
time:   00:12:47
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285256/

[7285257] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2}
time:   00:31:03
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285257/

[7285258] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}}
time:   00:34:52
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285258/

[7285259] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag}
time:   00:16:04
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285259/

[7285260] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-stupid} supported-random-distro$/{centos_8} tasks/crash}
time:   00:18:08
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285260/

[7285261] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{centos_latest}}
time:   00:46:04
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285261/

[7285262] rados/rest/{mgr-restful supported-random-distro$/{centos_8}}
time:   00:16:02
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285262/

[7285265] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest}
time:   00:22:03
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285265/

[7285266] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados tasks/mon_recovery validater/lockdep}
time:   00:20:27
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285266/

[7285268] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}}
time:   00:15:38
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285268/

[7285270] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{centos_8}}
time:   00:13:59
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285270/

[7285271] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:17:25
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285271/

[7285272] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/one workloads/snaps-few-objects}
time:   00:40:54
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285272/

[7285273] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest}}
time:   00:14:04
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285273/

[7285274] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews}
time:   00:14:29
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285274/

[7285275] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites}
time:   00:24:08
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285275/

[7285276] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait}
time:   00:15:17
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285276/

[7285278] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_rgw_multisite}
time:   00:19:11
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285278/

[7285279] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-bitmap} supported-random-distro$/{centos_latest} tasks/failover}
time:   00:23:05
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285279/

[7285280] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest}}
time:   00:13:42
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285280/

[7285282] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:18:22
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285282/

[7285283] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/cache-snaps}
time:   00:36:20
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285283/

[7285285] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:18:19
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285285/

[7285286] rados/objectstore/{backends/fusestore supported-random-distro$/{centos_8}}
time:   00:14:11
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285286/

[7285287] rados/singleton/{all/mon-config mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}}
time:   00:15:15
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285287/

[7285288] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest}}
time:   00:20:20
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285288/

[7285289] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{centos_latest} tasks/insights}
time:   00:18:57
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285289/

[7285290] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{centos_8}}
time:   00:12:29
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285290/

[7285291] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1}
time:   00:31:37
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285291/

[7285292] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}}
time:   00:22:26
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285292/

[7285293] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:30:54
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285293/

[7285294] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2}
time:   00:29:33
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285294/

[7285295] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest}}
time:   00:34:08
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285295/

[7285296] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{centos_8}}
time:   00:14:02
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285296/

[7285297] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep}
time:   00:19:01
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285297/

[7285298] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:20:37
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285298/

[7285299] rados/singleton/{all/peer mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest}}
time:   00:11:00
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285299/

[7285300] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{centos_8}}
time:   00:14:18
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285300/

[7285301] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/radosbench}
time:   01:36:49
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285301/

[7285302] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/small-objects}
time:   00:25:07
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285302/

[7285303] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/sync workloads/rados_5925}
time:   00:21:36
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285303/

[7285305] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}}
time:   00:23:50
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285305/

[7285306] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{centos_8} tasks/module_selftest}
time:   00:32:19
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285306/

[7285307] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest}}
time:   00:21:48
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285307/

[7285308] rados/standalone/{supported-random-distro$/{centos_latest} workloads/mgr}
time:   00:21:35
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285308/

[7285309] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{centos_latest}}
time:   00:17:07
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285309/

[7285310] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-radosbench}
time:   02:25:18
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285310/

[7285311] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/snaps-few-objects-localized}
time:   00:30:39
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285311/

[7285312] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}}
time:   00:20:28
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285312/

[7285313] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{centos_latest}}
time:   00:20:26
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285313/

[7285316] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites}
time:   00:33:15
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285316/

[7285317] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:21:26
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285317/

[7285318] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/snaps-few-objects}
time:   00:28:06
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285318/

[7285319] rados/singleton/{all/radostool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest}}
time:   00:17:17
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285319/

[7285320] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} tasks/rados_workunit_loadgen_big}
time:   00:31:52
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285320/

[7285321] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{centos_latest}}
time:   00:26:23
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285321/

[7285322] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/rados_api_tests}
time:   00:29:47
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285322/

[7285323] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-stupid} supported-random-distro$/{centos_latest} tasks/per_module_finisher_stats}
time:   00:11:16
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285323/

[7285324] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{centos_latest}}
time:   00:19:15
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285324/

[7285325] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}}
time:   00:13:23
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285325/

[7285327] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/admin_socket_objecter_requests}
time:   00:17:01
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285327/

[7285328] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/connectivity start}
time:   00:15:11
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285328/

[7285329] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-small-objects-balanced}
time:   00:21:16
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285329/

[7285330] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:25:38
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285330/

[7285331] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache-agent-big}
time:   00:19:32
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285331/

[7285332] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{centos_8}}
time:   00:30:03
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285332/

[7285333] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep}
time:   00:34:14
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285333/

[7285334] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest}}
time:   00:12:15
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285334/

[7285335] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}
time:   00:17:09
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285335/

[7285336] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait}
time:   00:14:15
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285336/

[7285338] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites}
time:   00:26:23
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285338/

[7285339] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy}
time:   00:29:29
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285339/

[7285340] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start}
time:   00:15:16
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285340/

[7285341] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} tasks/mon_recovery}
time:   00:21:58
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285341/

[7285343] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_rgw_multisite}
time:   00:17:22
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285343/

[7285344] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps}
time:   00:34:49
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285344/

[7285346] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{centos_8}}
time:   00:15:12
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285346/

[7285347] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-fast-read}
time:   00:23:54
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285347/

[7285348] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:28:58
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285348/

[7285349] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/quincy backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/snaps-few-objects}
time:   00:38:21
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285349/

[7285350] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}
time:   00:14:37
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285350/

[7285351] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rmdir-reactivate}
time:   00:14:39
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285351/

[7285352] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{centos_8}}
time:   00:13:56
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285352/

[7285354] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest}}
time:   00:49:53
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285354/

[7285355] rados/objectstore/{backends/objectstore-bluestore-a supported-random-distro$/{centos_latest}}
time:   02:13:49
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285355/

[7285356] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{centos_8}}
time:   00:12:17
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285356/

[7285357] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_set_mon_crush_locations}
time:   00:18:22
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285357/

[7285358] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}}
time:   00:22:46
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285358/

[7285359] rados/standalone/{supported-random-distro$/{centos_8} workloads/mon}
time:   01:07:27
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285359/

[7285360] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/cache}
time:   00:20:14
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285360/

[7285361] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{centos_8}}
time:   00:10:39
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285361/

[7285362] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites}
time:   00:24:47
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285362/

[7285363] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}}
time:   00:23:04
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285363/

[7285364] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/repave-all}
time:   00:13:47
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285364/

[7285365] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-many-deletes}
time:   00:21:52
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285365/

[7285366] rados/singleton-nomsgr/{all/crushdiff mon_election/classic rados supported-random-distro$/{centos_8}}
time:   00:12:05
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285366/

[7285367] rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}}
time:   00:11:28
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285367/

[7285368] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest}
time:   00:22:28
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285368/

[7285369] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep}
time:   00:15:16
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285369/

[7285371] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/test_rbd_api}
time:   00:21:28
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285371/

[7285372] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/pool-snaps-few-objects}
time:   00:25:45
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285372/

[7285373] rados/singleton/{all/admin-socket mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest}}
time:   00:10:13
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285373/

[7285374] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/sync-many workloads/pool-create-delete}
time:   00:15:56
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285374/

[7285375] rados/objectstore/{backends/objectstore-bluestore-b supported-random-distro$/{centos_8}}
time:   00:09:54
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285375/

[7285376] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/rados_api_tests}
time:   00:29:14
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285376/

[7285377] rados/singleton/{all/backfill-toofull mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest}}
time:   00:26:30
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285377/

[7285378] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} tasks/scrub_test}
time:   00:11:44
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285378/

[7285379] rados/singleton/{all/deduptool mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}}
time:   00:15:21
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285379/

[7285380] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1}
time:   00:24:52
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285380/

[7285381] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects}
time:   00:24:41
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285381/

[7285382] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity random-objectstore$/{bluestore-hybrid} supported-random-distro$/{centos_8} tasks/crash}
time:   00:22:36
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285382/

[7285383] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}}
time:   00:15:20
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285383/

[7285384] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites}
time:   00:25:31
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285384/

[7285385] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start}
time:   00:15:26
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285385/

[7285386] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/radosbench}
time:   01:04:42
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285386/

[7285388] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/rgw 3-final}
time:   00:13:31
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285388/

[7285389] rados/singleton/{all/divergent_priors2 mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}}
time:   00:13:33
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285389/

[7285390] rados/standalone/{supported-random-distro$/{centos_8} workloads/osd}
time:   02:37:20
info:   http://pulpito.ceph.com/yuriw-2023-05-24_18:46:05-rados-wip-yuri2-testing-2023-05-24-0747-distro-default-smithi/7285390/