2023-06-15T01:30:54.960 INFO:root:teuthology version: 0.0.1.dev124+g961f4fb 2023-06-15T01:30:54.961 INFO:teuthology.results:seed: '3431' 2023-06-15T01:52:06.162 INFO:root:teuthology version: 0.0.1.dev124+g961f4fb 2023-06-15T01:52:06.166 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-06-15T05:08:21.751 INFO:teuthology.results:Tests finished! gathering results... 2023-06-15T05:08:23.261 INFO:scrape:Found 20 jobs 2023-06-15T05:08:23.353 INFO:scrape:Found 1 distinct failure reasons 2023-06-15T05:08:23.354 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rbd/crimson/test_crimson_librbd.sh 2023-06-15T05:08:23.355 INFO:scrape:1 jobs: ['7304948'] 2023-06-15T05:08:23.355 INFO:scrape:suites: ['clusters/fixed-1', 'crimson-rados/rbd/{centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'tasks/rbd_api_tests}'] 2023-06-15T05:08:23.355 INFO:scrape: 2023-06-15T05:08:23.357 INFO:teuthology.results:Sending results to yingxin.cheng@intel.com: Test Run: yingxin-2023-06-15_01:29:33-crimson-rados-wip-yingxin-msgr-multi-core-base-crimson-6-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yingxin-2023-06-15_01:29:33-crimson-rados-wip-yingxin-msgr-multi-core-base-crimson-6-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-15_01:29:33-crimson-rados-wip-yingxin-msgr-multi-core-base-crimson-6-distro-default-smithi/ failed: 1 dead: 0 running: 0 waiting: 0 queued: 0 passed: 19 Fail ================================================================= [7304948] crimson-rados/rbd/{centos_latest clusters/fixed-1 crimson_qa_overrides deploy/ceph tasks/rbd_api_tests} ----------------------------------------------------------------- time: 03:22:52 info: http://pulpito.ceph.com/yingxin-2023-06-15_01:29:33-crimson-rados-wip-yingxin-msgr-multi-core-base-crimson-6-distro-default-smithi/7304948/ log: http://qa-proxy.ceph.com/teuthology/yingxin-2023-06-15_01:29:33-crimson-rados-wip-yingxin-msgr-multi-core-base-crimson-6-distro-default-smithi/7304948/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8235db06ff524aa9b1052d0c537c5bf1 Command failed (workunit test rbd/crimson/test_crimson_librbd.sh) on smithi086 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4e4479c675ce10581c8d93b4ec914e139c24514b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=61 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/crimson/test_crims on_librbd.sh' Pass ================================================================= [7304947] crimson-rados/basic/{centos_latest clusters/fixed-2 crimson_qa_overrides deploy/ceph tasks/rados_api_tests} time: 00:33:27 info: http://pulpito.ceph.com/yingxin-2023-06-15_01:29:33-crimson-rados-wip-yingxin-msgr-multi-core-base-crimson-6-distro-default-smithi/7304947/ [7304949] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/admin_socket_objecter_requests} time: 00:19:40 info: http://pulpito.ceph.com/yingxin-2023-06-15_01:29:33-crimson-rados-wip-yingxin-msgr-multi-core-base-crimson-6-distro-default-smithi/7304949/ [7304950] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/pool-snaps-few-objects} time: 00:33:24 info: http://pulpito.ceph.com/yingxin-2023-06-15_01:29:33-crimson-rados-wip-yingxin-msgr-multi-core-base-crimson-6-distro-default-smithi/7304950/ [7304951] crimson-rados/rbd/{centos_latest clusters/fixed-1 crimson_qa_overrides deploy/ceph tasks/rbd_api_tests_old_format} time: 00:25:53 info: http://pulpito.ceph.com/yingxin-2023-06-15_01:29:33-crimson-rados-wip-yingxin-msgr-multi-core-base-crimson-6-distro-default-smithi/7304951/ [7304952] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/radosbench-high-concurrency} time: 00:32:54 info: http://pulpito.ceph.com/yingxin-2023-06-15_01:29:33-crimson-rados-wip-yingxin-msgr-multi-core-base-crimson-6-distro-default-smithi/7304952/ [7304953] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/radosbench} time: 00:31:50 info: http://pulpito.ceph.com/yingxin-2023-06-15_01:29:33-crimson-rados-wip-yingxin-msgr-multi-core-base-crimson-6-distro-default-smithi/7304953/ [7304954] crimson-rados/basic/{centos_latest clusters/fixed-2 crimson_qa_overrides deploy/ceph tasks/rados_python} time: 00:21:12 info: http://pulpito.ceph.com/yingxin-2023-06-15_01:29:33-crimson-rados-wip-yingxin-msgr-multi-core-base-crimson-6-distro-default-smithi/7304954/ [7304955] crimson-rados/rbd/{centos_latest clusters/fixed-1 crimson_qa_overrides deploy/ceph tasks/rbd_cls_tests} time: 00:21:41 info: http://pulpito.ceph.com/yingxin-2023-06-15_01:29:33-crimson-rados-wip-yingxin-msgr-multi-core-base-crimson-6-distro-default-smithi/7304955/ [7304956] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/small-objects-balanced} time: 00:28:42 info: http://pulpito.ceph.com/yingxin-2023-06-15_01:29:33-crimson-rados-wip-yingxin-msgr-multi-core-base-crimson-6-distro-default-smithi/7304956/ [7304957] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/small-objects-localized} time: 00:28:11 info: http://pulpito.ceph.com/yingxin-2023-06-15_01:29:33-crimson-rados-wip-yingxin-msgr-multi-core-base-crimson-6-distro-default-smithi/7304957/ [7304958] crimson-rados/rbd/{centos_latest clusters/fixed-1 crimson_qa_overrides deploy/ceph tasks/rbd_lock_and_fence} time: 00:19:28 info: http://pulpito.ceph.com/yingxin-2023-06-15_01:29:33-crimson-rados-wip-yingxin-msgr-multi-core-base-crimson-6-distro-default-smithi/7304958/ [7304959] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/small-objects} time: 00:25:38 info: http://pulpito.ceph.com/yingxin-2023-06-15_01:29:33-crimson-rados-wip-yingxin-msgr-multi-core-base-crimson-6-distro-default-smithi/7304959/ [7304960] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/snaps-few-objects-balanced} time: 00:33:48 info: http://pulpito.ceph.com/yingxin-2023-06-15_01:29:33-crimson-rados-wip-yingxin-msgr-multi-core-base-crimson-6-distro-default-smithi/7304960/ [7304961] crimson-rados/basic/{centos_latest clusters/fixed-2 crimson_qa_overrides deploy/ceph tasks/readwrite} time: 00:20:10 info: http://pulpito.ceph.com/yingxin-2023-06-15_01:29:33-crimson-rados-wip-yingxin-msgr-multi-core-base-crimson-6-distro-default-smithi/7304961/ [7304962] crimson-rados/rbd/{centos_latest clusters/fixed-1 crimson_qa_overrides deploy/ceph tasks/rbd_python_api_tests} time: 00:36:55 info: http://pulpito.ceph.com/yingxin-2023-06-15_01:29:33-crimson-rados-wip-yingxin-msgr-multi-core-base-crimson-6-distro-default-smithi/7304962/ [7304963] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/snaps-few-objects-localized} time: 00:35:11 info: http://pulpito.ceph.com/yingxin-2023-06-15_01:29:33-crimson-rados-wip-yingxin-msgr-multi-core-base-crimson-6-distro-default-smithi/7304963/ [7304964] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/snaps-few-objects} time: 00:33:26 info: http://pulpito.ceph.com/yingxin-2023-06-15_01:29:33-crimson-rados-wip-yingxin-msgr-multi-core-base-crimson-6-distro-default-smithi/7304964/ [7304965] crimson-rados/rbd/{centos_latest clusters/fixed-1 crimson_qa_overrides deploy/ceph tasks/rbd_python_api_tests_old_format} time: 00:31:10 info: http://pulpito.ceph.com/yingxin-2023-06-15_01:29:33-crimson-rados-wip-yingxin-msgr-multi-core-base-crimson-6-distro-default-smithi/7304965/ [7304966] crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/write_fadvise_dontneed} time: 00:25:14 info: http://pulpito.ceph.com/yingxin-2023-06-15_01:29:33-crimson-rados-wip-yingxin-msgr-multi-core-base-crimson-6-distro-default-smithi/7304966/