2022-07-08T21:03:09.305 INFO:root:teuthology version: 1.1.1.dev381+g10062088 2022-07-08T21:03:09.306 INFO:teuthology.results:seed: '7466' 2022-07-08T21:36:50.145 INFO:root:teuthology version: 1.1.1.dev381+g10062088 2022-07-08T21:36:50.168 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-07-09T03:59:25.665 INFO:teuthology.results:Tests finished! gathering results... 2022-07-09T03:59:28.971 INFO:scrape:Found 30 jobs 2022-07-09T03:59:29.336 INFO:scrape:Found 2 distinct failure reasons 2022-07-09T03:59:29.338 INFO:scrape:Failure: Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400000 --objects 1024 --max-in-flight 64 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 50 --op write_excl 0 --op append_excl 50 --pool unique_pool_0' 2022-07-09T03:59:29.338 INFO:scrape:13 jobs: ['6920891', '6920901', '6920900', '6920912', '6920897', '6920889', '6920910', '6920904', '6920905', '6920893', '6920898', '6920896', '6920888'] 2022-07-09T03:59:29.339 INFO:scrape:suites intersection: ['clusters/{fixed-2', 'fast/fast', 'mon_election/classic', 'msgr-failures/osd-delay', 'objectstore/bluestore-bitmap', 'openstack}', 'rados', 'rados:thrash-erasure-code/{ceph', 'recovery-overrides/{more-partial-recovery}', 'supported-random-distro$/{rhel_8}', 'thrashers/minsize_recovery', 'thrashosds-health', 'workloads/ec-small-objects-balanced}'] 2022-07-09T03:59:29.339 INFO:scrape:suites union: ['clusters/{fixed-2', 'fast/fast', 'mon_election/classic', 'msgr-failures/osd-delay', 'objectstore/bluestore-bitmap', 'openstack}', 'rados', 'rados:thrash-erasure-code/{ceph', 'recovery-overrides/{more-partial-recovery}', 'supported-random-distro$/{rhel_8}', 'thrashers/minsize_recovery', 'thrashosds-health', 'workloads/ec-small-objects-balanced}'] 2022-07-09T03:59:29.340 INFO:scrape: 2022-07-09T03:59:29.340 INFO:scrape:Failure: {'smithi050.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} 2022-07-09T03:59:29.341 INFO:scrape:3 jobs: ['6920907', '6920914', '6920890'] 2022-07-09T03:59:29.341 INFO:scrape:suites intersection: ['clusters/{fixed-2', 'fast/fast', 'mon_election/classic', 'msgr-failures/osd-delay', 'objectstore/bluestore-bitmap', 'openstack}', 'rados', 'rados:thrash-erasure-code/{ceph', 'recovery-overrides/{more-partial-recovery}', 'supported-random-distro$/{rhel_8}', 'thrashers/minsize_recovery', 'thrashosds-health', 'workloads/ec-small-objects-balanced}'] 2022-07-09T03:59:29.341 INFO:scrape:suites union: ['clusters/{fixed-2', 'fast/fast', 'mon_election/classic', 'msgr-failures/osd-delay', 'objectstore/bluestore-bitmap', 'openstack}', 'rados', 'rados:thrash-erasure-code/{ceph', 'recovery-overrides/{more-partial-recovery}', 'supported-random-distro$/{rhel_8}', 'thrashers/minsize_recovery', 'thrashosds-health', 'workloads/ec-small-objects-balanced}'] 2022-07-09T03:59:29.343 INFO:scrape: 2022-07-09T03:59:29.344 INFO:teuthology.results:Sending results to ksirivad@redhat.com: Test Run: ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/ failed: 13 dead: 3 running: 0 waiting: 0 queued: 0 passed: 14 Fail ================================================================= [6920888] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} ----------------------------------------------------------------- time: 06:48:19 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920888/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920888/ Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400000 --objects 1024 --max-in-flight 64 --size 4000000 --min- stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 50 --op write_excl 0 --op append_excl 50 --pool unique_pool_0' [6920889] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} ----------------------------------------------------------------- time: 06:47:53 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920889/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920889/ Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400000 --objects 1024 --max-in-flight 64 --size 4000000 --min- stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 50 --op write_excl 0 --op append_excl 50 --pool unique_pool_0' [6920891] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} ----------------------------------------------------------------- time: 06:47:43 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920891/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920891/ Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400000 --objects 1024 --max-in-flight 64 --size 4000000 --min- stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 50 --op write_excl 0 --op append_excl 50 --pool unique_pool_0' [6920893] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} ----------------------------------------------------------------- time: 06:46:01 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920893/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920893/ Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400000 --objects 1024 --max-in-flight 64 --size 4000000 --min- stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 50 --op write_excl 0 --op append_excl 50 --pool unique_pool_0' [6920896] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} ----------------------------------------------------------------- time: 06:44:00 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920896/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920896/ Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400000 --objects 1024 --max-in-flight 64 --size 4000000 --min- stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 50 --op write_excl 0 --op append_excl 50 --pool unique_pool_0' [6920897] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} ----------------------------------------------------------------- time: 06:43:01 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920897/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920897/ Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400000 --objects 1024 --max-in-flight 64 --size 4000000 --min- stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 50 --op write_excl 0 --op append_excl 50 --pool unique_pool_0' [6920898] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} ----------------------------------------------------------------- time: 06:42:47 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920898/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920898/ Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400000 --objects 1024 --max-in-flight 64 --size 4000000 --min- stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 50 --op write_excl 0 --op append_excl 50 --pool unique_pool_0' [6920900] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} ----------------------------------------------------------------- time: 06:38:50 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920900/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920900/ Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400000 --objects 1024 --max-in-flight 64 --size 4000000 --min- stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 50 --op write_excl 0 --op append_excl 50 --pool unique_pool_0' [6920901] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} ----------------------------------------------------------------- time: 06:38:52 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920901/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920901/ Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400000 --objects 1024 --max-in-flight 64 --size 4000000 --min- stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 50 --op write_excl 0 --op append_excl 50 --pool unique_pool_0' [6920904] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} ----------------------------------------------------------------- time: 06:35:28 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920904/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920904/ Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400000 --objects 1024 --max-in-flight 64 --size 4000000 --min- stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 50 --op write_excl 0 --op append_excl 50 --pool unique_pool_0' [6920905] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} ----------------------------------------------------------------- time: 06:35:07 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920905/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920905/ Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400000 --objects 1024 --max-in-flight 64 --size 4000000 --min- stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 50 --op write_excl 0 --op append_excl 50 --pool unique_pool_0' [6920910] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} ----------------------------------------------------------------- time: 06:32:18 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920910/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920910/ Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400000 --objects 1024 --max-in-flight 64 --size 4000000 --min- stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 50 --op write_excl 0 --op append_excl 50 --pool unique_pool_0' [6920912] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} ----------------------------------------------------------------- time: 06:30:51 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920912/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920912/ Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400000 --objects 1024 --max-in-flight 64 --size 4000000 --min- stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 50 --op write_excl 0 --op append_excl 50 --pool unique_pool_0' Dead ================================================================= [6920890] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} ----------------------------------------------------------------- time: 00:07:26 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920890/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920890/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b141edbd9d6047509869ea9587b19132 {'smithi050.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} [6920907] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} ----------------------------------------------------------------- time: 00:07:26 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920907/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920907/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d7c3a059315f4b86a2e7c90959e18ce4 {'smithi050.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} [6920914] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} ----------------------------------------------------------------- time: 00:07:01 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920914/ log: http://qa-proxy.ceph.com/teuthology/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920914/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5f6e4119fdc54f8b910fd2d026dc72af {'smithi050.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} Pass ================================================================= [6920887] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 00:27:13 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920887/ [6920892] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 00:28:27 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920892/ [6920894] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 00:26:44 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920894/ [6920895] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 00:27:14 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920895/ [6920899] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 00:26:12 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920899/ [6920902] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 00:26:23 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920902/ [6920903] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 00:26:32 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920903/ [6920906] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 00:27:31 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920906/ [6920908] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 00:27:19 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920908/ [6920909] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 00:26:54 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920909/ [6920911] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 00:26:53 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920911/ [6920913] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 00:26:31 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920913/ [6920915] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 00:26:22 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920915/ [6920916] rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 00:26:43 info: http://pulpito.ceph.com/ksirivad-2022-07-08_20:45:16-rados:thrash-erasure-code-main-distro-default-smithi/6920916/