2024-02-02T02:27:31.803 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-02-02T02:27:31.803 INFO:teuthology.results:subset: '111/120000' 2024-02-02T02:27:31.803 INFO:teuthology.results:seed: '8014' 2024-02-02T17:00:14.379 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-02-02T17:00:14.399 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2024-02-03T03:44:14.390 INFO:teuthology.results:Tests finished! gathering results... 2024-02-03T03:44:39.517 INFO:scrape:Found 328 jobs 2024-02-03T03:44:44.148 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543071/teuthology.log 2024-02-03T03:44:44.148 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543071/teuthology.log 2024-02-03T03:44:44.148 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543071/teuthology.log 2024-02-03T03:44:49.828 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542965/teuthology.log 2024-02-03T03:44:49.828 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542965/teuthology.log 2024-02-03T03:44:49.855 INFO:scrape:Found 17 distinct failure reasons 2024-02-03T03:44:49.856 INFO:scrape:Dead: b'2024-02-02T16:39:07.538 DEBUG:teuthology.exit:Finished running handlers' 2024-02-03T03:44:49.856 INFO:scrape:44 jobs 2024-02-03T03:44:49.857 INFO:scrape:suites intersection: ['rados'] 2024-02-03T03:44:49.857 INFO:scrape:suites union: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{more-partial-recovery}', '3-scrub-overrides/{max-simultaneous-scrubs-3}', 'backoff/peering_and_degraded', 'ceph', 'cluster/{12-osds', 'clusters/{fixed-2', 'clusters/{fixed-4', 'crc-failures/bad_map_crc_failure', 'd-balancer/on', 'fast/fast', 'fast/normal', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/osd-delay', 'msgr-failures/osd-dispatch-delay', 'msgr/async', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-low-osd-mem-target', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/singleton-nomsgr/{all/admin_socket_output', 'rados/thrash-erasure-code-big/{ceph', 'rados/thrash-erasure-code-isa/{arch/x86_64', 'rados/thrash-erasure-code-overwrites/{bluestore-bitmap', 'rados/thrash-erasure-code-shec/{ceph', 'rados/thrash-erasure-code/{ceph', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'recovery-overrides/{default}', 'recovery-overrides/{more-active-recovery}', 'recovery-overrides/{more-async-partial-recovery}', 'recovery-overrides/{more-async-recovery}', 'recovery-overrides/{more-partial-recovery}', 'supported-random-distro$/{centos_latest}', 'supported-random-distro$/{centos_latest}}', 'supported-random-distro$/{ubuntu_latest}', 'supported-random-distro$/{ubuntu_latest}}', 'thrashers/careful', 'thrashers/default', 'thrashers/fastread', 'thrashers/mapgap', 'thrashers/minsize_recovery', 'thrashers/morepggrow', 'thrashers/pggrow', 'thrashosds-health', 'workloads/cache-agent-big}', 'workloads/ec-pool-snaps-few-objects-overwrites}', 'workloads/ec-rados-plugin=clay-k=4-m=2}', 'workloads/ec-rados-plugin=isa-k=2-m=1}', 'workloads/ec-rados-plugin=jerasure-k=2-m=1}', 'workloads/ec-rados-plugin=jerasure-k=3-m=1}', 'workloads/ec-rados-plugin=jerasure-k=4-m=2}', 'workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}', 'workloads/ec-rados-plugin=shec-k=4-m=3-c=2}', 'workloads/ec-small-objects-balanced}', 'workloads/ec-small-objects-fast-read-overwrites}', 'workloads/ec-small-objects-fast-read}', 'workloads/ec-small-objects-many-deletes}', 'workloads/ec-small-objects-overwrites}', 'workloads/ec-small-objects}', 'workloads/ec-snaps-few-objects-overwrites}'] 2024-02-03T03:44:49.857 INFO:scrape: 2024-02-03T03:44:49.857 INFO:scrape:Failure: Command failed (workunit test cephadm/test_cephadm_timeout.py) on smithi060 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e10f4b28ec4e5fd19161c663ab66bd3d25b667cc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm_timeout.py' 2024-02-03T03:44:49.857 INFO:scrape:1 jobs: ['7543037'] 2024-02-03T03:44:49.857 INFO:scrape:suites: ['agent/on', 'mon_election/connectivity', 'rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm_timeout}'] 2024-02-03T03:44:49.857 INFO:scrape: 2024-02-03T03:44:49.857 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x179) [0x5571c0abbf85] 2024-02-03T03:44:49.858 INFO:scrape:ceph version 19.0.0-1243-ge10f4b28 (e10f4b28ec4e5fd19161c663ab66bd3d25b667cc) squid (dev) 1: /lib64/libc.so.6(+0x54db0) [0x7f0798e54db0] 2: /lib64/libc.so.6(+0xa154c) [0x7f0798ea154c] 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x179) [0x5571c0abbf85] 6: ceph-osd(+0x3f30e6) [0x5571c0abc0e6] 7: ceph-osd(+0x39b274) [0x5571c0a64274] 8: (ECBackend::RecoveryBackend::continue_recovery_op(ECBackend::RecoveryBackend::RecoveryOp&, RecoveryMessages*)+0x129f) [0x5571c0fad9ff] 9: (ECBackend::RecoveryBackend::run_recovery_op(ECRecoveryHandle&, int)+0xf83) [0x5571c0fb0123] 10: (ECBackend::run_recovery_op(PGBackend::RecoveryHandle*, int)+0x23) [0x5571c0fb0533] 11: (PrimaryLogPG::recover_backfill(unsigned long, ThreadPool::TPHandle&, bool*)+0x2845) [0x5571c0d70fe5] 12: (PrimaryLogPG::start_recovery_ops(unsigned long, ThreadPool::TPHandle&, unsigned long*)+0xf89) [0x5571c0d62de9] 13: (OSD::do_recovery(PG*, unsigned int, unsigned long, int, ThreadPool::TPHandle&)+0x245) [0x5571c0c46375] 14: (ceph::osd::scheduler::PGRecovery::run(OSD*, OSDShard*, boost::intrusive_ptr&, ThreadPool::TPHandle&)+0xca) [0x5571c0e91bba] 15: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xd07) [0x5571c0c626f7] 16: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x2aa) [0x5571c1160a8a] 17: ceph-osd(+0xa98034) [0x5571c1161034] 18: /lib64/libc.so.6(+0x9f802) [0x7f0798e9f802] 19: /lib64/libc.so.6(+0x3f450) [0x7f0798e3f450] 2024-02-03T03:44:49.858 INFO:scrape:1 jobs: ['7542849'] 2024-02-03T03:44:49.858 INFO:scrape:suites: ['mon_election/classic', 'msgr-failures/many', 'msgr/async-v1only', 'objectstore/bluestore-comp-zlib', 'rados', 'rados/singleton/{all/backfill-toofull', 'supported-random-distro$/{centos_latest}}'] 2024-02-03T03:44:49.858 INFO:scrape: 2024-02-03T03:44:49.858 INFO:scrape:Failure: reached maximum tries (51) after waiting for 300 seconds 2024-02-03T03:44:49.858 INFO:scrape:2 jobs: ['7543151', '7543032'] 2024-02-03T03:44:49.858 INFO:scrape:suites intersection: [] 2024-02-03T03:44:49.858 INFO:scrape:suites union: ['2-inject-leak/osd', 'centos_latest}', 'clusters/{fixed-2', 'fast/normal', 'mon_election/connectivity', 'msgr-failures/few', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/thrash-erasure-code/{ceph', 'rados/valgrind-leaks/{1-start', 'recovery-overrides/{more-partial-recovery}', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/careful', 'thrashosds-health', 'workloads/ec-radosbench}'] 2024-02-03T03:44:49.858 INFO:scrape: 2024-02-03T03:44:49.858 INFO:scrape:Timeout 6h running clone.client.0/qa/workunits/rados/test.sh 2024-02-03T03:44:49.859 INFO:scrape:2 jobs: ['7542860', '7543076'] 2024-02-03T03:44:49.859 INFO:scrape:suites intersection: ['ceph', 'clusters/{fixed-2', 'd-thrash/default/{default', 'mon_election/connectivity', 'msgr-failures/few', 'msgr/async', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_api_tests', 'thrashosds-health}', 'validater/lockdep}'] 2024-02-03T03:44:49.859 INFO:scrape:suites union: ['ceph', 'clusters/{fixed-2', 'd-thrash/default/{default', 'mon_election/connectivity', 'msgr-failures/few', 'msgr/async', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-hybrid', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_api_tests', 'thrashosds-health}', 'validater/lockdep}'] 2024-02-03T03:44:49.859 INFO:scrape: 2024-02-03T03:44:49.859 INFO:scrape:Timeout 120 running ceph --cluster ceph osd dump --format=json 2024-02-03T03:44:49.859 INFO:scrape:2 jobs: ['7542843', '7543144'] 2024-02-03T03:44:49.859 INFO:scrape:suites intersection: ['clusters/{fixed-2', 'mon_election/classic', 'msgr-failures/many', 'msgr/async-v2only', 'openstack}', 'rados', 'rados/basic/{ceph', 'supported-random-distro$/{centos_latest}', 'tasks/repair_test}'] 2024-02-03T03:44:49.859 INFO:scrape:suites union: ['clusters/{fixed-2', 'mon_election/classic', 'msgr-failures/many', 'msgr/async-v2only', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/basic/{ceph', 'supported-random-distro$/{centos_latest}', 'tasks/repair_test}'] 2024-02-03T03:44:49.859 INFO:scrape: 2024-02-03T03:44:49.859 INFO:scrape:Failure: valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun 2024-02-03T03:44:49.860 INFO:scrape:5 jobs: ['7542893', '7543041', '7543114', '7542970', '7543040'] 2024-02-03T03:44:49.860 INFO:scrape:suites intersection: [] 2024-02-03T03:44:49.860 INFO:scrape:suites union: ['2-inject-leak/none', 'centos_latest}', 'ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/valgrind-leaks/{1-start', 'rados/verify/{centos_latest', 'tasks/mon_recovery', 'tasks/rados_api_tests', 'tasks/rados_cls_all', 'validater/valgrind}'] 2024-02-03T03:44:49.860 INFO:scrape: 2024-02-03T03:44:49.860 INFO:scrape:Timeout 3h running clone.client.0/qa/standalone/osd/divergent-priors.sh 2024-02-03T03:44:49.860 INFO:scrape:1 jobs: ['7542865'] 2024-02-03T03:44:49.860 INFO:scrape:suites: ['rados/standalone/{supported-random-distro$/{centos_latest}', 'workloads/osd}'] 2024-02-03T03:44:49.860 INFO:scrape: 2024-02-03T03:44:49.860 INFO:scrape:Failure: Command failed (workunit test osd-backfill/osd-backfill-space.sh) on smithi132 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e10f4b28ec4e5fd19161c663ab66bd3d25b667cc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd-backfill/osd-backfill-space.sh' 2024-02-03T03:44:49.861 INFO:scrape:1 jobs: ['7543160'] 2024-02-03T03:44:49.861 INFO:scrape:suites: ['rados/standalone/{supported-random-distro$/{centos_latest}', 'workloads/osd-backfill}'] 2024-02-03T03:44:49.861 INFO:scrape: 2024-02-03T03:44:49.861 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x16e) [0x5592a1cee09c] 2024-02-03T03:44:49.861 INFO:scrape:ceph version 19.0.0-1243-ge10f4b28 (e10f4b28ec4e5fd19161c663ab66bd3d25b667cc) squid (dev) 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fd8e6238520] 2: pthread_kill() 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x16e) [0x5592a1cee09c] 6: ceph-osd(+0x3f61fd) [0x5592a1cee1fd] 7: ceph-osd(+0x39f004) [0x5592a1c97004] 8: (ECBackend::RecoveryBackend::continue_recovery_op(ECBackend::RecoveryBackend::RecoveryOp&, RecoveryMessages*)+0x12cf) [0x5592a21cb71f] 9: (ECBackend::RecoveryBackend::run_recovery_op(ECRecoveryHandle&, int)+0xe4c) [0x5592a21cdbbc] 10: (ECBackend::run_recovery_op(PGBackend::RecoveryHandle*, int)+0x23) [0x5592a21cdfc3] 11: (PrimaryLogPG::recover_replicas(unsigned long, ThreadPool::TPHandle&, bool*)+0x12f2) [0x5592a1f88f52] 12: (PrimaryLogPG::start_recovery_ops(unsigned long, ThreadPool::TPHandle&, unsigned long*)+0x25b) [0x5592a1f9499b] 13: (OSD::do_recovery(PG*, unsigned int, unsigned long, int, ThreadPool::TPHandle&)+0x245) [0x5592a1e5fd85] 14: (ceph::osd::scheduler::PGRecovery::run(OSD*, OSDShard*, boost::intrusive_ptr&, ThreadPool::TPHandle&)+0xca) [0x5592a20bec3a] 15: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xab3) [0x5592a1e81543] 16: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x293) [0x5592a237a6a3] 17: ceph-osd(+0xa82c04) [0x5592a237ac04] 18: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7fd8e628ab43] 19: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7fd8e631ca00] 2024-02-03T03:44:49.861 INFO:scrape:1 jobs: ['7542901'] 2024-02-03T03:44:49.861 INFO:scrape:suites: ['mon_election/classic', 'msgr-failures/many', 'msgr/async-v1only', 'objectstore/bluestore-comp-lz4', 'rados', 'rados/singleton/{all/ec-lost-unfound', 'supported-random-distro$/{ubuntu_latest}}'] 2024-02-03T03:44:49.861 INFO:scrape: 2024-02-03T03:44:49.861 INFO:scrape:Failure: Command failed (workunit test scrub/osd-scrub-repair.sh) on smithi160 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e10f4b28ec4e5fd19161c663ab66bd3d25b667cc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh' 2024-02-03T03:44:49.861 INFO:scrape:1 jobs: ['7542899'] 2024-02-03T03:44:49.861 INFO:scrape:suites: ['rados/standalone/{supported-random-distro$/{centos_latest}', 'workloads/scrub}'] 2024-02-03T03:44:49.861 INFO:scrape: 2024-02-03T03:44:49.862 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rados/test.sh 2024-02-03T03:44:49.862 INFO:scrape:2 jobs: ['7542904', '7543082'] 2024-02-03T03:44:49.862 INFO:scrape:suites intersection: ['ceph', 'clusters/{fixed-2', 'openstack}', 'rados', 'supported-random-distro$/{centos_latest}', 'thrashosds-health', 'workloads/rados_api_tests}'] 2024-02-03T03:44:49.862 INFO:scrape:suites union: ['1-pg-log-overrides/normal_pg_log', '1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{default}', '2-recovery-overrides/{more-partial-recovery}', '3-scrub-overrides/{max-simultaneous-scrubs-1}', '3-scrub-overrides/{max-simultaneous-scrubs-5}', 'backoff/normal', 'backoff/peering_and_degraded', 'ceph', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'crc-failures/default', 'd-balancer/read', 'd-balancer/upmap-read', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/osd-dispatch-delay', 'msgr/async', 'msgr/async-v1only', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{centos_latest}', 'thrashers/careful', 'thrashers/pggrow', 'thrashosds-health', 'workloads/rados_api_tests}'] 2024-02-03T03:44:49.862 INFO:scrape: 2024-02-03T03:44:49.862 INFO:scrape:Failure: Command failed on smithi106 with status 5: 'sudo systemctl stop ceph-088a7ef6-c187-11ee-95b6-87774f69a715@mon.b' 2024-02-03T03:44:49.862 INFO:scrape:1 jobs: ['7542969'] 2024-02-03T03:44:49.862 INFO:scrape:suites: ['0-size-min-size-overrides/2-size-2-min-size', '1-install/nautilus-v1only', 'backoff/peering', 'ceph', 'clusters/{openstack', 'd-balancer/crush-compat', 'mon_election/classic', 'msgr-failures/few', 'rados', 'rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools}', 'thrashers/mapgap', 'thrashosds-health', 'three-plus-one}', 'workloads/radosbench}'] 2024-02-03T03:44:49.862 INFO:scrape: 2024-02-03T03:44:49.862 INFO:scrape:Dead: None 2024-02-03T03:44:49.863 INFO:scrape:2 jobs: ['7543071', '7542965'] 2024-02-03T03:44:49.863 INFO:scrape: 2024-02-03T03:44:49.863 INFO:scrape:Failure: Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_lazy_omap_stats' 2024-02-03T03:44:49.863 INFO:scrape:2 jobs: ['7542862', '7543028'] 2024-02-03T03:44:49.863 INFO:scrape:suites intersection: ['rados', 'rados/singleton-nomsgr/{all/lazy_omap_stats_output'] 2024-02-03T03:44:49.863 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados', 'rados/singleton-nomsgr/{all/lazy_omap_stats_output', 'supported-random-distro$/{centos_latest}}', 'supported-random-distro$/{ubuntu_latest}}'] 2024-02-03T03:44:49.863 INFO:scrape: 2024-02-03T03:44:49.863 INFO:scrape:Failure: wait_for_clean: failed before timeout expired 2024-02-03T03:44:49.863 INFO:scrape:1 jobs: ['7542891'] 2024-02-03T03:44:49.863 INFO:scrape:suites: ['mon_election/connectivity', 'msgr-failures/few', 'msgr/async', 'objectstore/bluestore-bitmap', 'rados', 'rados/singleton/{all/ec-inconsistent-hinfo', 'supported-random-distro$/{ubuntu_latest}}'] 2024-02-03T03:44:49.863 INFO:scrape: 2024-02-03T03:44:49.863 INFO:scrape:Failure: Command failed (workunit test erasure-code/test-erasure-code.sh) on smithi196 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e10f4b28ec4e5fd19161c663ab66bd3d25b667cc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh' 2024-02-03T03:44:49.864 INFO:scrape:1 jobs: ['7542999'] 2024-02-03T03:44:49.864 INFO:scrape:suites: ['rados/standalone/{supported-random-distro$/{centos_latest}', 'workloads/erasure-code}'] 2024-02-03T03:44:49.864 INFO:scrape: 2024-02-03T03:44:49.876 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi ================================================================= info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/ failed: 24 dead: 46 running: 0 waiting: 0 queued: 0 passed: 258 Fail ================================================================= [7542843] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} tasks/repair_test} ----------------------------------------------------------------- time: 00:26:45 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542843/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542843/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2379b290915142e7928fa7dd37b30710 Command failed on smithi064 with status 124: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' [7542849] rados/singleton/{all/backfill-toofull mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest}} ----------------------------------------------------------------- time: 00:21:23 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542849/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542849/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5f06aaed1d4545bca8193ad0c25632b0 Command failed on smithi170 with status 124: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' [7542860] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados tasks/rados_api_tests validater/lockdep} ----------------------------------------------------------------- time: 06:14:20 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542860/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542860/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8ef2c7add58547488f7197dabd6efc27 Command failed (workunit test rados/test.sh) on smithi031 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e10f4b28ec4e5fd19161c663ab66bd3d25b667cc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 ALLOW_TIMEOUTS=1 adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [7542862] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{centos_latest}} ----------------------------------------------------------------- time: 00:08:44 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542862/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542862/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b5eb31f87a1846c8b7a9a5f4488bfafd Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_lazy_omap_stats' [7542865] rados/standalone/{supported-random-distro$/{centos_latest} workloads/osd} ----------------------------------------------------------------- time: 03:24:33 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542865/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542865/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7c98a4dd5ef945cf9fc1e0c84c026823 Command failed (workunit test osd/divergent-priors.sh) on smithi057 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e10f4b28ec4e5fd19161c663ab66bd3d25b667cc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh' [7542891] rados/singleton/{all/ec-inconsistent-hinfo mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:52:39 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542891/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542891/ sentry: https://sentry.ceph.com/organizations/ceph/?query=20b219dc7d194def98096c3a86e4938a wait_for_clean: failed before timeout expired [7542893] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados tasks/rados_cls_all validater/valgrind} ----------------------------------------------------------------- time: 01:45:09 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542893/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542893/ valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun [7542899] rados/standalone/{supported-random-distro$/{centos_latest} workloads/scrub} ----------------------------------------------------------------- time: 00:34:44 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542899/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542899/ sentry: https://sentry.ceph.com/organizations/ceph/?query=aba1a8fc28de49a3bc88d4b32dfb4b2c Command failed (workunit test scrub/osd-scrub-repair.sh) on smithi160 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e10f4b28ec4e5fd19161c663ab66bd3d25b667cc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub- repair.sh' [7542901] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:20:21 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542901/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542901/ sentry: https://sentry.ceph.com/organizations/ceph/?query=623437ba272c4fa7ab5f466348d200db Command failed on smithi123 with status 124: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' [7542904] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/rados_api_tests} ----------------------------------------------------------------- time: 03:28:35 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542904/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542904/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9e49382c8cea4c2e83acbe3f2b2a6ff3 Command failed (workunit test rados/test.sh) on smithi045 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e10f4b28ec4e5fd19161c663ab66bd3d25b667cc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [7542969] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/radosbench} ----------------------------------------------------------------- time: 00:12:01 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542969/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542969/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7408e338cb084cf5b2b94c809e864d67 Command failed on smithi106 with status 5: 'sudo systemctl stop ceph-088a7ef6-c187-11ee-95b6-87774f69a715@mon.b' [7542970] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} ----------------------------------------------------------------- time: 00:59:45 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542970/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542970/ valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun [7542999] rados/standalone/{supported-random-distro$/{centos_latest} workloads/erasure-code} ----------------------------------------------------------------- time: 00:23:34 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542999/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542999/ sentry: https://sentry.ceph.com/organizations/ceph/?query=52d1f143cf8b44f0805dd2a87084966f Command failed (workunit test erasure-code/test-erasure-code.sh) on smithi196 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e10f4b28ec4e5fd19161c663ab66bd3d25b667cc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test- erasure-code.sh' [7543028] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:09:50 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543028/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543028/ sentry: https://sentry.ceph.com/organizations/ceph/?query=05cd3ec2c37e47159498991a65e0755e Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_lazy_omap_stats' [7543032] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-radosbench} ----------------------------------------------------------------- time: 01:37:03 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543032/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543032/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d6d1d7b3f3c44f82b13d0286b0aa8c30 reached maximum tries (801) after waiting for 4800 seconds [7543037] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_cephadm_timeout} ----------------------------------------------------------------- time: 00:16:48 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543037/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543037/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5c15c70353b54a2ba29c375d41cb0e8b Command failed (workunit test cephadm/test_cephadm_timeout.py) on smithi060 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e10f4b28ec4e5fd19161c663ab66bd3d25b667cc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_cephadm_timeout.py' [7543040] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} ----------------------------------------------------------------- time: 00:17:24 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543040/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543040/ valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun [7543041] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} ----------------------------------------------------------------- time: 00:31:02 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543041/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543041/ valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun [7543076] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} ----------------------------------------------------------------- time: 06:13:25 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543076/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543076/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6f840f5ff44e4a40a2b4a5bfbe8b56c1 Command failed (workunit test rados/test.sh) on smithi110 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e10f4b28ec4e5fd19161c663ab66bd3d25b667cc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 ALLOW_TIMEOUTS=1 adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [7543082] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/rados_api_tests} ----------------------------------------------------------------- time: 03:14:17 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543082/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543082/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fe793d1090734bbf9ad077fdede3e53f Command failed (workunit test rados/test.sh) on smithi122 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e10f4b28ec4e5fd19161c663ab66bd3d25b667cc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [7543114] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} ----------------------------------------------------------------- time: 01:47:31 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543114/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543114/ valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun [7543144] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} tasks/repair_test} ----------------------------------------------------------------- time: 00:26:40 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543144/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543144/ sentry: https://sentry.ceph.com/organizations/ceph/?query=50d747dc33dd4c7eb5fa094739018ca9 Command failed on smithi099 with status 124: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' [7543151] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} ----------------------------------------------------------------- time: 00:21:30 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543151/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543151/ reached maximum tries (51) after waiting for 300 seconds [7543160] rados/standalone/{supported-random-distro$/{centos_latest} workloads/osd-backfill} ----------------------------------------------------------------- time: 01:25:04 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543160/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543160/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d3295eede3b242d98ea5e7cc3362e0ee Command failed (workunit test osd-backfill/osd-backfill-space.sh) on smithi132 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e10f4b28ec4e5fd19161c663ab66bd3d25b667cc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd-backfill/osd- backfill-space.sh' Dead ================================================================= [7542838] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542838/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542838/ hit max job timeout [7542851] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-small-objects-many-deletes} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542851/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542851/ hit max job timeout [7542855] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542855/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542855/ hit max job timeout [7542858] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-snaps-few-objects-overwrites} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542858/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542858/ hit max job timeout [7542863] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542863/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542863/ hit max job timeout [7542878] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542878/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542878/ hit max job timeout [7542885] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542885/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542885/ hit max job timeout [7542889] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542889/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542889/ hit max job timeout [7542898] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542898/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542898/ hit max job timeout [7542903] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542903/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542903/ hit max job timeout [7542911] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542911/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542911/ hit max job timeout [7542919] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542919/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542919/ hit max job timeout [7542924] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542924/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542924/ hit max job timeout [7542932] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{centos_latest}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542932/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542932/ hit max job timeout [7542938] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542938/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542938/ hit max job timeout [7542951] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542951/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542951/ hit max job timeout [7542953] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542953/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542953/ hit max job timeout [7542959] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542959/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542959/ hit max job timeout [7542962] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542962/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542962/ hit max job timeout [7542965] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/crush} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542965/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542965/ Error reimaging machines: Failed to power on smithi191 [7542974] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542974/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542974/ hit max job timeout [7542985] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542985/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542985/ hit max job timeout [7542994] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542994/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542994/ hit max job timeout [7542996] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542996/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542996/ hit max job timeout [7543000] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543000/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543000/ hit max job timeout [7543007] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543007/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543007/ [7543012] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/cache-agent-big} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543012/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543012/ hit max job timeout [7543023] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543023/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543023/ [7543043] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543043/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543043/ [7543044] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543044/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543044/ [7543059] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543059/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543059/ [7543067] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-small-objects-balanced} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543067/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543067/ [7543068] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543068/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543068/ [7543071] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mix} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543071/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543071/ Error reimaging machines: Failed to power on smithi046 [7543077] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543077/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543077/ [7543087] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543087/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543087/ [7543094] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543094/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543094/ [7543097] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543097/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543097/ hit max job timeout [7543107] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-fast-read} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543107/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543107/ [7543108] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543108/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543108/ [7543113] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543113/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543113/ [7543131] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543131/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543131/ [7543136] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543136/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543136/ [7543140] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543140/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543140/ [7543141] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{default} supported-random-distro$/{centos_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-many-deletes} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543141/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543141/ [7543150] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543150/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543150/ Pass ================================================================= [7542835] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:10:37 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542835/ [7542836] rados/objectstore/{backends/objectstore-bluestore-b supported-random-distro$/{ubuntu_latest}} time: 00:08:39 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542836/ [7542837] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} tasks/mon_clock_with_skews} time: 00:07:17 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542837/ [7542839] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{centos_latest} tasks/progress} time: 00:19:21 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542839/ [7542840] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/cache-agent-small} time: 00:13:13 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542840/ [7542841] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:17:11 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542841/ [7542842] rados/singleton/{all/admin-socket mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}} time: 00:07:12 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542842/ [7542844] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:09:34 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542844/ [7542845] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}} time: 00:30:42 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542845/ [7542846] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:17:37 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542846/ [7542847] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/pacific backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/snaps-few-objects} time: 00:40:22 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542847/ [7542848] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:25:01 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542848/ [7542850] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-zlib} tasks/e2e} time: 00:26:52 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542850/ [7542852] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:14:26 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542852/ [7542853] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_ca_signed_key} time: 00:16:06 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542853/ [7542854] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:08:27 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542854/ [7542856] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache-pool-snaps} time: 00:20:25 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542856/ [7542857] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/rgw 3-final} time: 00:14:57 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542857/ [7542859] rados/singleton/{all/deduptool mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest}} time: 00:09:53 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542859/ [7542861] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/cache-snaps-balanced} time: 00:22:30 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542861/ [7542864] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_cephadm} time: 00:16:53 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542864/ [7542866] rados/singleton/{all/divergent_priors mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:11:32 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542866/ [7542867] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:19:32 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542867/ [7542868] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-snaps} time: 00:22:38 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542868/ [7542869] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} tasks/scrub_test} time: 00:10:10 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542869/ [7542870] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_latest} tasks/prometheus} time: 00:12:48 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542870/ [7542871] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/force-sync-many workloads/rados_mon_osdmap_prune} time: 00:42:22 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542871/ [7542872] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:12:01 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542872/ [7542873] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_cephadm_repos} time: 00:07:34 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542873/ [7542874] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:12:33 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542874/ [7542875] rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:12:17 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542875/ [7542876] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/cache} time: 00:16:07 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542876/ [7542877] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} tasks/mon_recovery} time: 00:15:21 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542877/ [7542879] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} time: 00:16:49 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542879/ [7542880] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:19:39 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542880/ [7542881] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/dedup-io-mixed} time: 00:12:53 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542881/ [7542882] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:11:35 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542882/ [7542883] rados/objectstore/{backends/objectstore-memstore supported-random-distro$/{centos_latest}} time: 00:07:10 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542883/ [7542884] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_cephadm_timeout} time: 00:23:30 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542884/ [7542886] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/quincy backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/test_rbd_api} time: 00:23:23 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542886/ [7542887] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:20:07 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542887/ [7542888] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/dedup-io-snaps} time: 00:11:48 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542888/ [7542890] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:17:09 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542890/ [7542892] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} tasks/libcephsqlite} time: 00:11:51 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542892/ [7542894] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_extra_daemon_features} time: 00:22:27 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542894/ [7542895] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:13:09 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542895/ [7542896] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/pool-snaps-few-objects} time: 00:25:34 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542896/ [7542897] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:20:25 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542897/ [7542900] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-stupid} supported-random-distro$/{centos_latest} tasks/workunits} time: 00:08:37 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542900/ [7542902] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:18:18 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542902/ [7542905] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:07:58 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542905/ [7542906] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_host_drain} time: 00:21:50 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542906/ [7542907] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_mon_workunits} time: 00:30:21 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542907/ [7542908] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:09:22 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542908/ [7542909] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/radosbench-high-concurrency} time: 00:30:37 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542909/ [7542910] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:09:58 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542910/ [7542912] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:24:08 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542912/ [7542913] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:25:43 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542913/ [7542914] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest}} time: 00:36:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542914/ [7542915] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/radosbench} time: 01:14:17 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542915/ [7542916] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:15:57 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542916/ [7542917] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:13:47 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542917/ [7542918] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/rados_api_tests} time: 00:29:25 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542918/ [7542920] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:11:20 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542920/ [7542921] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:15:54 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542921/ [7542922] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:33:19 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542922/ [7542923] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/redirect} time: 00:13:41 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542923/ [7542925] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest}} time: 00:27:56 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542925/ [7542926] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/reef backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/cache-snaps} time: 00:34:58 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542926/ [7542927] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_monitoring_stack_basic} time: 00:21:06 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542927/ [7542928] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-snappy} tasks/dashboard} time: 01:25:19 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542928/ [7542929] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-bitmap} supported-random-distro$/{centos_latest} tasks/crash} time: 00:10:04 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542929/ [7542930] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{centos_latest}} time: 00:35:41 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542930/ [7542931] rados/rest/{mgr-restful supported-random-distro$/{ubuntu_latest}} time: 00:14:20 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542931/ [7542933] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/c2c} time: 00:15:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542933/ [7542934] rados/upgrade/parallel/{0-random-distro$/{centos_8.stream_container_tools} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:49:34 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542934/ [7542935] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:21:26 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542935/ [7542936] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados tasks/mon_recovery validater/lockdep} time: 00:11:44 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542936/ [7542937] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/redirect_promote_tests} time: 00:18:49 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542937/ [7542939] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest}} time: 00:08:07 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542939/ [7542940] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:16:53 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542940/ [7542941] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/redirect_set_object} time: 00:10:41 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542941/ [7542942] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:11:36 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542942/ [7542943] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/basic 3-final} time: 00:14:01 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542943/ [7542944] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:13:52 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542944/ [7542945] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/snaps-few-objects} time: 00:35:14 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542945/ [7542946] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest}} time: 00:12:22 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542946/ [7542947] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/rados_cls_all} time: 00:21:49 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542947/ [7542948] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_orch_cli} time: 00:25:08 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542948/ [7542949] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/set-chunks-read} time: 00:16:30 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542949/ [7542950] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} tasks/mon_clock_with_skews} time: 00:07:52 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542950/ [7542952] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:09:06 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542952/ [7542954] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:15:34 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542954/ [7542955] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:14:24 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542955/ [7542956] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/small-objects-balanced} time: 00:19:03 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542956/ [7542957] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:10:39 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542957/ [7542958] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_orch_cli_mon} time: 00:31:43 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542958/ [7542960] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/small-objects-localized} time: 00:20:09 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542960/ [7542961] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 00:12:16 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542961/ [7542963] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{ubuntu_latest} tasks/failover} time: 00:15:26 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542963/ [7542964] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:17:35 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542964/ [7542966] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:14:54 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542966/ [7542967] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:09:26 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542967/ [7542968] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/small-objects} time: 00:19:08 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542968/ [7542971] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_rgw_multisite} time: 00:19:13 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542971/ [7542972] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} tasks/rados_python} time: 00:13:23 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542972/ [7542973] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:12:09 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542973/ [7542975] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_set_mon_crush_locations} time: 00:19:00 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542975/ [7542976] rados/singleton-nomsgr/{all/ceph-snapmapper mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:10:09 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542976/ [7542977] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects-balanced} time: 00:24:49 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542977/ [7542978] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:16:01 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542978/ [7542979] rados/objectstore/{backends/fusestore supported-random-distro$/{ubuntu_latest}} time: 00:09:56 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542979/ [7542980] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:19:16 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542980/ [7542981] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/sync-many workloads/pool-create-delete} time: 00:16:15 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542981/ [7542982] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/snaps-few-objects-localized} time: 00:22:36 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542982/ [7542983] rados/singleton-nomsgr/{all/crushdiff mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:11:03 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542983/ [7542984] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} tasks/mon_recovery} time: 00:22:44 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542984/ [7542986] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_adoption} time: 00:11:40 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542986/ [7542987] rados/singleton/{all/mon-config mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:15:31 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542987/ [7542988] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:13:55 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542988/ [7542989] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/snaps-few-objects} time: 00:28:35 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542989/ [7542990] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:20:03 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542990/ [7542991] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:09:47 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542991/ [7542992] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{ubuntu_latest} tasks/insights} time: 00:13:49 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542992/ [7542993] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/rados_stress_watch} time: 00:21:19 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542993/ [7542995] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest}} time: 00:15:44 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542995/ [7542997] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/write_fadvise_dontneed} time: 00:16:23 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542997/ [7542998] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_ca_signed_key} time: 00:22:32 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7542998/ [7543001] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:07:59 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543001/ [7543002] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:18:38 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543002/ [7543003] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest}} time: 00:30:36 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543003/ [7543004] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/admin_socket_objecter_requests} time: 00:31:58 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543004/ [7543005] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest}} time: 00:21:45 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543005/ [7543006] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep} time: 00:16:23 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543006/ [7543008] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_cephadm} time: 00:23:02 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543008/ [7543009] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v2only backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/rbd_cls} time: 00:24:44 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543009/ [7543010] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:11:57 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543010/ [7543011] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:14:06 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543011/ [7543013] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:14:29 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543013/ [7543014] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:18:22 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543014/ [7543015] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-comp-lz4} tasks/e2e} time: 00:28:22 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543015/ [7543016] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} thrashers/sync workloads/rados_5925} time: 00:10:53 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543016/ [7543017] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/cache-agent-small} time: 00:15:52 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543017/ [7543018] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:10:50 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543018/ [7543019] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_cephadm_repos} time: 00:07:13 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543019/ [7543020] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} tasks/rados_striper} time: 00:07:28 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543020/ [7543021] rados/singleton/{all/peer mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:10:35 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543021/ [7543022] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} time: 00:09:48 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543022/ [7543024] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-stupid} supported-random-distro$/{ubuntu_latest} tasks/module_selftest} time: 00:25:50 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543024/ [7543025] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools} 1-start 2-services/rgw 3-final} time: 00:15:16 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543025/ [7543026] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:17:27 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543026/ [7543027] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{ubuntu_latest}} time: 00:15:46 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543027/ [7543029] rados/standalone/{supported-random-distro$/{centos_latest} workloads/mgr} time: 00:14:25 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543029/ [7543030] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest}} time: 00:15:58 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543030/ [7543031] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:14:38 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543031/ [7543033] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/cache-pool-snaps} time: 00:19:29 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543033/ [7543034] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:12:07 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543034/ [7543035] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:22:20 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543035/ [7543036] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:09:39 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543036/ [7543038] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}} time: 00:15:46 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543038/ [7543039] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/cache-snaps-balanced} time: 00:22:46 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543039/ [7543042] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:24:02 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543042/ [7543045] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:14:06 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543045/ [7543046] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_big} time: 00:32:47 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543046/ [7543047] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/cache-snaps} time: 00:22:24 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543047/ [7543048] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_extra_daemon_features} time: 00:15:38 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543048/ [7543049] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}} time: 00:07:43 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543049/ [7543050] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/snaps-few-objects} time: 00:35:18 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543050/ [7543051] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:20:01 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543051/ [7543052] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_host_drain} time: 00:23:13 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543052/ [7543053] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/rados_api_tests} time: 00:44:28 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543053/ [7543054] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache} time: 00:17:43 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543054/ [7543055] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{centos_latest} tasks/per_module_finisher_stats} time: 00:08:44 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543055/ [7543056] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:22:43 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543056/ [7543057] rados/singleton/{all/radostool mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:19:21 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543057/ [7543058] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:09:33 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543058/ [7543060] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:18:30 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543060/ [7543061] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/dedup-io-mixed} time: 00:13:25 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543061/ [7543062] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:18:10 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543062/ [7543063] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/misc} time: 01:10:45 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543063/ [7543064] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:19:28 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543064/ [7543065] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest}} time: 00:35:35 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543065/ [7543066] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/dedup-io-snaps} time: 00:13:31 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543066/ [7543069] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:20:42 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543069/ [7543070] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:08:15 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543070/ [7543072] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{centos_latest}} time: 00:23:17 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543072/ [7543073] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest}} time: 00:12:58 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543073/ [7543074] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/pool-snaps-few-objects} time: 00:29:45 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543074/ [7543075] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_monitoring_stack_basic} time: 00:24:15 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543075/ [7543078] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:25:38 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543078/ [7543079] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:14:24 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543079/ [7543080] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:17:17 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543080/ [7543081] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:18:15 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543081/ [7543083] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:34:22 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543083/ [7543084] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_latest} tasks/progress} time: 00:23:21 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543084/ [7543085] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:19:55 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543085/ [7543086] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:09:06 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543086/ [7543088] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/radosbench-high-concurrency} time: 00:52:14 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543088/ [7543089] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_mon_osdmap_prune} time: 00:58:33 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543089/ [7543090] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/test_rbd_api} time: 00:35:56 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543090/ [7543091] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest}} time: 00:08:10 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543091/ [7543092] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_orch_cli} time: 00:24:24 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543092/ [7543093] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:16:18 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543093/ [7543095] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} tasks/rados_workunit_loadgen_mostlyread} time: 00:26:31 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543095/ [7543096] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} tasks/dashboard} time: 01:24:11 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543096/ [7543098] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/mon-stretch} time: 00:25:29 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543098/ [7543099] rados/upgrade/parallel/{0-random-distro$/{centos_8.stream_container_tools_crun} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:49:45 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543099/ [7543100] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/radosbench} time: 01:15:29 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543100/ [7543101] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_orch_cli_mon} time: 00:29:57 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543101/ [7543102] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest}} time: 00:09:29 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543102/ [7543103] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:12:36 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543103/ [7543104] rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_rhel8} 1-start 2-services/basic 3-final} time: 00:13:56 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543104/ [7543105] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/redirect} time: 00:19:04 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543105/ [7543106] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:08:54 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543106/ [7543109] rados/singleton/{all/test-noautoscale-flag mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:10:51 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543109/ [7543110] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:16:26 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543110/ [7543111] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/redirect_promote_tests} time: 00:13:39 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543111/ [7543112] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:11:07 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543112/ [7543115] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:18:22 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543115/ [7543116] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{ubuntu_latest} tasks/prometheus} time: 00:12:57 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543116/ [7543117] rados/singleton/{all/thrash-backfill-full mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest}} time: 01:58:17 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543117/ [7543118] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/redirect_set_object} time: 00:11:21 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543118/ [7543119] rados/objectstore/{backends/objectstore-bluestore-a supported-random-distro$/{ubuntu_latest}} time: 02:19:59 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543119/ [7543120] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_rgw_multisite} time: 00:22:56 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543120/ [7543121] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/readwrite} time: 00:14:34 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543121/ [7543122] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:08:19 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543122/ [7543123] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:14:25 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543123/ [7543124] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/set-chunks-read} time: 00:13:34 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543124/ [7543125] rados/singleton/{all/thrash-eio mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest}} time: 00:50:27 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543125/ [7543126] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_set_mon_crush_locations} time: 00:23:29 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543126/ [7543127] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/one workloads/rados_mon_workunits} time: 00:31:12 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543127/ [7543128] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/mon} time: 01:06:44 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543128/ [7543129] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:07:37 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543129/ [7543130] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest} tasks/mon_clock_no_skews} time: 00:07:38 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543130/ [7543132] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/cache-snaps} time: 00:37:53 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543132/ [7543133] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_adoption} time: 00:10:56 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543133/ [7543134] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/small-objects-balanced} time: 00:25:44 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543134/ [7543135] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:21:43 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543135/ [7543137] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/repave-all} time: 00:14:38 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543137/ [7543138] rados/singleton-nomsgr/{all/ceph-snapmapper mon_election/classic rados supported-random-distro$/{centos_latest}} time: 00:07:44 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543138/ [7543139] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap-read mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/small-objects-localized} time: 00:18:58 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543139/ [7543142] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_latest}} time: 00:18:39 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543142/ [7543143] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:16:03 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543143/ [7543145] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/small-objects} time: 00:22:02 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543145/ [7543146] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:13:53 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543146/ [7543147] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{centos_latest} tasks/workunits} time: 00:08:53 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543147/ [7543148] rados/singleton-nomsgr/{all/crushdiff mon_election/connectivity rados supported-random-distro$/{centos_latest}} time: 00:08:24 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543148/ [7543149] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_ca_signed_key} time: 00:14:20 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543149/ [7543152] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} time: 00:11:27 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543152/ [7543153] rados/singleton/{all/watch-notify-same-primary mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} time: 00:10:25 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543153/ [7543154] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/snaps-few-objects-balanced} time: 00:22:53 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543154/ [7543155] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_cephadm} time: 00:16:59 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543155/ [7543156] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:09:45 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543156/ [7543157] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:24:45 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543157/ [7543158] rados/singleton/{all/admin-socket mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:09:50 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543158/ [7543159] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/snaps-few-objects-localized} time: 00:26:28 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543159/ [7543161] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:34:24 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543161/ [7543162] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/sync-many workloads/pool-create-delete} time: 00:16:45 info: https://pulpito.ceph.com/yuriw-2024-02-01_21:56:37-rados-wip-yuri5-testing-2024-01-31-1657-distro-default-smithi/7543162/