2022-04-07T09:51:30.309 INFO:root:teuthology version: 1.1.0-3dd79763 2022-04-07T09:51:30.311 INFO:teuthology.results:seed: '9719' 2022-04-07T10:22:37.040 INFO:root:teuthology version: 1.1.0-3dd79763 2022-04-07T10:22:37.070 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-04-07T11:22:41.435 INFO:teuthology.results:Tests finished! gathering results... 2022-04-07T11:22:42.698 INFO:scrape:Found 21 jobs 2022-04-07T11:22:49.195 INFO:scrape:Found 4 distinct failure reasons 2022-04-07T11:22:49.196 INFO:scrape:Crash: Found coredumps on ubuntu@smithi092.front.sepia.ceph.com 2022-04-07T11:22:49.197 INFO:scrape:ceph version 17.0.0-11492-g15b299a2 (15b299a20032eda013329b86822fb63ac50c9bb7) quincy (dev) 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x143c0) [0x7f4f2f40f3c0] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b7) [0x7f4f2f971834] 5: /usr/lib/ceph/libceph-common.so.2(+0x27b9de) [0x7f4f2f9719de] 6: (rbd::mirror::NamespaceReplayer::shut_down_remote_pool_watcher(rbd::mirror::Peer&, Context*)+0x1f5) [0x55cd099bf2f5] 7: (Context::complete(int)+0xd) [0x55cd0994d2cd] 8: (boost::asio::detail::completion_handler, 0u> >::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long)+0x5a) [0x55cd0994d76a] 9: (boost::asio::detail::strand_service::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long)+0x85) [0x55cd09addf65] 10: /lib/librados.so.2(+0xdf248) [0x7f4f3066d248] 11: /lib/librados.so.2(+0xe4eea) [0x7f4f30672eea] 12: /lib/x86_64-linux-gnu/libstdc++.so.6(+0xd6de4) [0x7f4f2f2e9de4] 13: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f4f2f403609] 14: clone() 2022-04-07T11:22:49.197 INFO:scrape:5 jobs: ['6780838', '6780829', '6780849', '6780847', '6780832'] 2022-04-07T11:22:49.197 INFO:scrape:suites intersection: ['cluster/3-nodes/{base/{install', 'extra/{rbd-mirror}', 'mirror', 'msgr-failures/few', 'rbd/mirror-thrash/{base/{openstack}', 'roles}', 'workloads/rbd-mirror-journal-workunit-multipeer}'] 2022-04-07T11:22:49.198 INFO:scrape:suites union: ['cluster/3-nodes/{base/{install', 'extra/{rbd-mirror}', 'mirror', 'msgr-failures/few', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-low-osd-mem-target', 'policy/none', 'policy/simple', 'rbd/mirror-thrash/{base/{openstack}', 'roles}', 'supported-random-distro$/{centos_8}}', 'supported-random-distro$/{ubuntu_latest}}', 'workloads/rbd-mirror-journal-workunit-multipeer}'] 2022-04-07T11:22:49.198 INFO:scrape: 2022-04-07T11:22:49.199 INFO:scrape:Failure: Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi176 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=15b299a20032eda013329b86822fb63ac50c9bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\' RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 RBD_MIRROR_USE_RBD_MIRROR=1 USE_CLUSTER3=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal.sh' 2022-04-07T11:22:49.199 INFO:scrape:3 jobs: ['6780843', '6780845', '6780834'] 2022-04-07T11:22:49.200 INFO:scrape:suites intersection: ['cluster/3-nodes/{base/{install', 'extra/{rbd-mirror}', 'mirror', 'msgr-failures/few', 'rbd/mirror-thrash/{base/{openstack}', 'roles}', 'workloads/rbd-mirror-journal-workunit-multipeer}'] 2022-04-07T11:22:49.200 INFO:scrape:suites union: ['cluster/3-nodes/{base/{install', 'extra/{rbd-mirror}', 'mirror', 'msgr-failures/few', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-comp-zstd', 'policy/none', 'policy/simple', 'rbd/mirror-thrash/{base/{openstack}', 'roles}', 'supported-random-distro$/{centos_8}}', 'supported-random-distro$/{ubuntu_latest}}', 'workloads/rbd-mirror-journal-workunit-multipeer}'] 2022-04-07T11:22:49.201 INFO:scrape: 2022-04-07T11:22:49.201 INFO:scrape:Failure: Command failed (workunit test rbd/rbd_mirror_snapshot.sh) on smithi165 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=15b299a20032eda013329b86822fb63ac50c9bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\' RBD_IMAGE_FEATURES=layering,exclusive-lock,object-map,fast-diff RBD_MIRROR_CONFIG_KEY=1 RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 USE_CLUSTER3=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_snapshot.sh' 2022-04-07T11:22:49.201 INFO:scrape:1 jobs: ['6780837'] 2022-04-07T11:22:49.202 INFO:scrape:suites: ['cluster/3-nodes/{base/{install', 'extra/{mirror-extra}', 'mirror', 'msgr-failures/few', 'objectstore/bluestore-comp-zstd', 'rbd/mirror/{base/{openstack}', 'roles}', 'supported-random-distro$/{ubuntu_latest}}', 'workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer}'] 2022-04-07T11:22:49.202 INFO:scrape: 2022-04-07T11:22:49.203 INFO:scrape:Failure: "2022-04-07T10:12:02.326604+0000 mon.a (mon.0) 99 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log 2022-04-07T11:22:49.203 INFO:scrape:2 jobs: ['6780836', '6780833'] 2022-04-07T11:22:49.203 INFO:scrape:suites intersection: ['cluster/3-nodes/{base/{install', 'mirror', 'msgr-failures/few', 'roles}', 'supported-random-distro$/{ubuntu_latest}}'] 2022-04-07T11:22:49.204 INFO:scrape:suites union: ['cluster/3-nodes/{base/{install', 'extra/{mirror-extra}', 'extra/{rbd-mirror}', 'mirror', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-hybrid', 'policy/none', 'rbd/mirror-thrash/{base/{openstack}', 'rbd/mirror/{base/{openstack}', 'roles}', 'supported-random-distro$/{ubuntu_latest}}', 'workloads/rbd-mirror-journal-workunit-multipeer}', 'workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer}'] 2022-04-07T11:22:49.204 INFO:scrape: 2022-04-07T11:22:49.206 INFO:teuthology.results:Sending results to arthur.outhenin-chalandre@cern.ch: Test Run: mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/ failed: 11 dead: 0 running: 0 waiting: 0 queued: 0 passed: 10 Fail ================================================================= [6780829] rbd/mirror-thrash/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit-multipeer} msgr-failures/few objectstore/bluestore-comp-lz4 policy/simple supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:55:55 info: http://pulpito.ceph.com/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780829/ log: http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780829/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bd3b4d6c70e847bc921b861a11efa3a4 Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi153 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=15b299a20032eda013329b86822fb63ac50c9bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\' RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 RBD_MIRROR_USE_RBD_MIRROR=1 USE_CLUSTER3=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal.sh' [6780832] rbd/mirror-thrash/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit-multipeer} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/simple supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:52:43 info: http://pulpito.ceph.com/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780832/ log: http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780832/ Found coredumps on ubuntu@smithi170.front.sepia.ceph.com [6780833] rbd/mirror/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:25:31 info: http://pulpito.ceph.com/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780833/ log: http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780833/ "2022-04-07T10:11:07.744502+0000 mon.a (mon.0) 239 : cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log [6780834] rbd/mirror-thrash/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit-multipeer} msgr-failures/few objectstore/bluestore-comp-snappy policy/simple supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:30:28 info: http://pulpito.ceph.com/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780834/ log: http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780834/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7275d76733264c57b8b4b221bd8805d5 Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi174 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=15b299a20032eda013329b86822fb63ac50c9bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\' RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 RBD_MIRROR_USE_RBD_MIRROR=1 USE_CLUSTER3=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal.sh' [6780836] rbd/mirror-thrash/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit-multipeer} msgr-failures/few objectstore/bluestore-bitmap policy/none supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:15:33 info: http://pulpito.ceph.com/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780836/ log: http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780836/ sentry: https://sentry.ceph.com/organizations/ceph/?query=41fea7a4e57a4b93b61eae15c1fd115a "2022-04-07T10:12:02.326604+0000 mon.a (mon.0) 99 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6780837] rbd/mirror/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:27:49 info: http://pulpito.ceph.com/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780837/ log: http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780837/ sentry: https://sentry.ceph.com/organizations/ceph/?query=923030827250453da699f8d9285b250a Command failed (workunit test rbd/rbd_mirror_snapshot.sh) on smithi165 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=15b299a20032eda013329b86822fb63ac50c9bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\' RBD_IMAGE_FEATURES=layering,exclusive-lock,object-map,fast-diff RBD_MIRROR_CONFIG_KEY=1 RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 USE_CLUSTER3=1 adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cep htest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_snapshot.sh' [6780838] rbd/mirror-thrash/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit-multipeer} msgr-failures/few objectstore/bluestore-comp-zlib policy/simple supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:51:17 info: http://pulpito.ceph.com/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780838/ log: http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780838/ Found coredumps on ubuntu@smithi092.front.sepia.ceph.com [6780843] rbd/mirror-thrash/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit-multipeer} msgr-failures/few objectstore/bluestore-comp-zstd policy/simple supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 01:05:24 info: http://pulpito.ceph.com/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780843/ log: http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780843/ sentry: https://sentry.ceph.com/organizations/ceph/?query=da7005f5b92f4b0dad15030c0467438d Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi176 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=15b299a20032eda013329b86822fb63ac50c9bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\' RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 RBD_MIRROR_USE_RBD_MIRROR=1 USE_CLUSTER3=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal.sh' [6780845] rbd/mirror-thrash/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit-multipeer} msgr-failures/few objectstore/bluestore-comp-snappy policy/none supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:26:53 info: http://pulpito.ceph.com/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780845/ log: http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780845/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8032d1420ed547d7a7089c0befb14be2 Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi050 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=15b299a20032eda013329b86822fb63ac50c9bb7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\' RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 RBD_MIRROR_USE_RBD_MIRROR=1 USE_CLUSTER3=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal.sh' [6780847] rbd/mirror-thrash/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit-multipeer} msgr-failures/few objectstore/bluestore-hybrid policy/simple supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:54:01 info: http://pulpito.ceph.com/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780847/ log: http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780847/ Found coredumps on ubuntu@smithi173.front.sepia.ceph.com [6780849] rbd/mirror-thrash/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit-multipeer} msgr-failures/few objectstore/bluestore-comp-zlib policy/none supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:48:48 info: http://pulpito.ceph.com/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780849/ log: http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780849/ Found coredumps on ubuntu@smithi151.front.sepia.ceph.com Pass ================================================================= [6780830] rbd/mirror/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8}} time: 00:26:24 info: http://pulpito.ceph.com/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780830/ [6780831] rbd/mirror-thrash/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit-multipeer} msgr-failures/few objectstore/filestore-xfs policy/none supported-random-distro$/{ubuntu_latest}} time: 00:45:15 info: http://pulpito.ceph.com/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780831/ [6780835] rbd/mirror/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_8}} time: 00:26:01 info: http://pulpito.ceph.com/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780835/ [6780839] rbd/mirror/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{centos_8}} time: 00:25:32 info: http://pulpito.ceph.com/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780839/ [6780840] rbd/mirror-thrash/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit-multipeer} msgr-failures/few objectstore/bluestore-comp-lz4 policy/none supported-random-distro$/{centos_8}} time: 00:53:49 info: http://pulpito.ceph.com/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780840/ [6780841] rbd/mirror/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest}} time: 00:34:35 info: http://pulpito.ceph.com/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780841/ [6780842] rbd/mirror-thrash/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit-multipeer} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/none supported-random-distro$/{centos_8}} time: 00:51:20 info: http://pulpito.ceph.com/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780842/ [6780844] rbd/mirror/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{centos_8}} time: 00:26:56 info: http://pulpito.ceph.com/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780844/ [6780846] rbd/mirror/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest}} time: 00:25:30 info: http://pulpito.ceph.com/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780846/ [6780848] rbd/mirror/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8}} time: 00:26:56 info: http://pulpito.ceph.com/mrfreezeex-2022-04-07_09:49:50-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6780848/