2023-07-10T06:42:16.707 INFO:root:teuthology version: 0.0.1.dev140+gb53f236 2023-07-10T06:42:16.708 INFO:teuthology.results:seed: '2293' 2023-07-10T07:29:23.128 INFO:root:teuthology version: 0.0.1.dev140+gb53f236 2023-07-10T07:29:23.159 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-07-10T10:25:37.108 INFO:teuthology.results:Tests finished! gathering results... 2023-07-10T10:25:39.126 INFO:scrape:Found 20 jobs 2023-07-10T10:25:39.555 INFO:scrape:Found 1 distinct failure reasons 2023-07-10T10:25:39.556 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rbd/crimson/test_crimson_librbd.sh 2023-07-10T10:25:39.557 INFO:scrape:1 jobs: ['7331965'] 2023-07-10T10:25:39.557 INFO:scrape:suites: ['clusters/fixed-1', 'crimson-rados/rbd/{centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'tasks/rbd_api_tests}'] 2023-07-10T10:25:39.557 INFO:scrape: 2023-07-10T10:25:39.558 INFO:teuthology.results:Sending results to yingxin.cheng@intel.com: Test Run: yingxin-2023-07-10_06:20:40-crimson-rados-wip-yingxin-multi-core-crimson-osd-cleanup-2-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yingxin-2023-07-10_06:20:40-crimson-rados-wip-yingxin-multi-core-crimson-osd-cleanup-2-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yingxin-2023-07-10_06:20:40-crimson-rados-wip-yingxin-multi-core-crimson-osd-cleanup-2-distro-default-smithi/ failed: 1 dead: 0 running: 0 waiting: 0 queued: 0 passed: 19 Fail ================================================================= [7331965] crimson-rados/rbd/{centos_latest clusters/fixed-1 crimson_qa_overrides deploy/ceph tasks/rbd_api_tests} ----------------------------------------------------------------- time: 03:16:00 info: http://pulpito.ceph.com/yingxin-2023-07-10_06:20:40-crimson-rados-wip-yingxin-multi-core-crimson-osd-cleanup-2-distro-default-smithi/7331965/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-07-10_06:20:40-crimson-rados-wip-yingxin-multi-core-crimson-osd-cleanup-2-distro-default-smithi/7331965/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8c3f56f4eb8d49f0a04f5bfb79b77f3a Command failed (workunit test rbd/crimson/test_crimson_librbd.sh) on smithi191 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8fe76c2999c5ed8a10870303e895bdc167e93837 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=61 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/crimson/test_crims on_librbd.sh' Pass ================================================================= [7331964] crimson-rados/basic/{centos_latest clusters/fixed-2 crimson_qa_overrides deploy/ceph tasks/rados_api_tests} time: 00:33:41 info: http://pulpito.ceph.com/yingxin-2023-07-10_06:20:40-crimson-rados-wip-yingxin-multi-core-crimson-osd-cleanup-2-distro-default-smithi/7331964/ [7331966] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/admin_socket_objecter_requests} time: 00:21:25 info: http://pulpito.ceph.com/yingxin-2023-07-10_06:20:40-crimson-rados-wip-yingxin-multi-core-crimson-osd-cleanup-2-distro-default-smithi/7331966/ [7331967] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/pool-snaps-few-objects} time: 00:37:19 info: http://pulpito.ceph.com/yingxin-2023-07-10_06:20:40-crimson-rados-wip-yingxin-multi-core-crimson-osd-cleanup-2-distro-default-smithi/7331967/ [7331968] crimson-rados/rbd/{centos_latest clusters/fixed-1 crimson_qa_overrides deploy/ceph tasks/rbd_api_tests_old_format} time: 00:27:32 info: http://pulpito.ceph.com/yingxin-2023-07-10_06:20:40-crimson-rados-wip-yingxin-multi-core-crimson-osd-cleanup-2-distro-default-smithi/7331968/ [7331969] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/radosbench-high-concurrency} time: 00:30:52 info: http://pulpito.ceph.com/yingxin-2023-07-10_06:20:40-crimson-rados-wip-yingxin-multi-core-crimson-osd-cleanup-2-distro-default-smithi/7331969/ [7331970] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/radosbench} time: 00:34:36 info: http://pulpito.ceph.com/yingxin-2023-07-10_06:20:40-crimson-rados-wip-yingxin-multi-core-crimson-osd-cleanup-2-distro-default-smithi/7331970/ [7331971] crimson-rados/basic/{centos_latest clusters/fixed-2 crimson_qa_overrides deploy/ceph tasks/rados_python} time: 00:28:54 info: http://pulpito.ceph.com/yingxin-2023-07-10_06:20:40-crimson-rados-wip-yingxin-multi-core-crimson-osd-cleanup-2-distro-default-smithi/7331971/ [7331972] crimson-rados/rbd/{centos_latest clusters/fixed-1 crimson_qa_overrides deploy/ceph tasks/rbd_cls_tests} time: 00:23:55 info: http://pulpito.ceph.com/yingxin-2023-07-10_06:20:40-crimson-rados-wip-yingxin-multi-core-crimson-osd-cleanup-2-distro-default-smithi/7331972/ [7331973] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/small-objects-balanced} time: 00:28:02 info: http://pulpito.ceph.com/yingxin-2023-07-10_06:20:40-crimson-rados-wip-yingxin-multi-core-crimson-osd-cleanup-2-distro-default-smithi/7331973/ [7331974] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/small-objects-localized} time: 00:26:05 info: http://pulpito.ceph.com/yingxin-2023-07-10_06:20:40-crimson-rados-wip-yingxin-multi-core-crimson-osd-cleanup-2-distro-default-smithi/7331974/ [7331975] crimson-rados/rbd/{centos_latest clusters/fixed-1 crimson_qa_overrides deploy/ceph tasks/rbd_lock_and_fence} time: 00:15:47 info: http://pulpito.ceph.com/yingxin-2023-07-10_06:20:40-crimson-rados-wip-yingxin-multi-core-crimson-osd-cleanup-2-distro-default-smithi/7331975/ [7331976] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/small-objects} time: 00:24:00 info: http://pulpito.ceph.com/yingxin-2023-07-10_06:20:40-crimson-rados-wip-yingxin-multi-core-crimson-osd-cleanup-2-distro-default-smithi/7331976/ [7331977] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/snaps-few-objects-balanced} time: 00:38:59 info: http://pulpito.ceph.com/yingxin-2023-07-10_06:20:40-crimson-rados-wip-yingxin-multi-core-crimson-osd-cleanup-2-distro-default-smithi/7331977/ [7331978] crimson-rados/basic/{centos_latest clusters/fixed-2 crimson_qa_overrides deploy/ceph tasks/readwrite} time: 00:18:13 info: http://pulpito.ceph.com/yingxin-2023-07-10_06:20:40-crimson-rados-wip-yingxin-multi-core-crimson-osd-cleanup-2-distro-default-smithi/7331978/ [7331979] crimson-rados/rbd/{centos_latest clusters/fixed-1 crimson_qa_overrides deploy/ceph tasks/rbd_python_api_tests} time: 00:38:09 info: http://pulpito.ceph.com/yingxin-2023-07-10_06:20:40-crimson-rados-wip-yingxin-multi-core-crimson-osd-cleanup-2-distro-default-smithi/7331979/ [7331980] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/snaps-few-objects-localized} time: 00:34:09 info: http://pulpito.ceph.com/yingxin-2023-07-10_06:20:40-crimson-rados-wip-yingxin-multi-core-crimson-osd-cleanup-2-distro-default-smithi/7331980/ [7331981] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/snaps-few-objects} time: 00:36:00 info: http://pulpito.ceph.com/yingxin-2023-07-10_06:20:40-crimson-rados-wip-yingxin-multi-core-crimson-osd-cleanup-2-distro-default-smithi/7331981/ [7331982] crimson-rados/rbd/{centos_latest clusters/fixed-1 crimson_qa_overrides deploy/ceph tasks/rbd_python_api_tests_old_format} time: 00:31:28 info: http://pulpito.ceph.com/yingxin-2023-07-10_06:20:40-crimson-rados-wip-yingxin-multi-core-crimson-osd-cleanup-2-distro-default-smithi/7331982/ [7331983] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/write_fadvise_dontneed} time: 00:21:03 info: http://pulpito.ceph.com/yingxin-2023-07-10_06:20:40-crimson-rados-wip-yingxin-multi-core-crimson-osd-cleanup-2-distro-default-smithi/7331983/