2022-02-03T10:29:12.559 INFO:root:teuthology version: 1.1.0-0f634dc1 2022-02-03T10:29:12.561 INFO:teuthology.results:seed: '80' 2022-02-03T10:31:05.801 INFO:root:teuthology version: 1.1.0-0f634dc1 2022-02-03T10:31:05.807 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-02-03T14:20:23.276 INFO:teuthology.results:Tests finished! gathering results... 2022-02-03T14:20:23.975 INFO:scrape:Found 6 jobs 2022-02-03T14:20:27.686 INFO:scrape:Found 3 distinct failure reasons 2022-02-03T14:20:27.687 INFO:scrape:Failure: "2022-02-03T10:59:06.239245+0000 mon.a (mon.0) 229 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log 2022-02-03T14:20:27.687 INFO:scrape:2 jobs: ['6658414', '6658415'] 2022-02-03T14:20:27.687 INFO:scrape:suites intersection: ['clusters/fixed-3', 'conf', 'install/ceph', 'krbd/rbd-nomount/{bluestore-bitmap', 'msgr-failures/many'] 2022-02-03T14:20:27.688 INFO:scrape:suites union: ['clusters/fixed-3', 'conf', 'install/ceph', 'krbd/rbd-nomount/{bluestore-bitmap', 'ms_mode/crc$/{crc}', 'ms_mode/legacy$/{legacy-rxbounce}', 'msgr-failures/many', 'tasks/krbd_latest_osdmap_on_map}', 'tasks/rbd_image_read}'] 2022-02-03T14:20:27.688 INFO:scrape: 2022-02-03T14:20:27.689 INFO:scrape:Failure: Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=nightly_pre-single-major 2022-02-03T14:20:27.689 INFO:scrape:1 jobs: ['6658410'] 2022-02-03T14:20:27.690 INFO:scrape:suites: ['clusters/separate-client', 'conf', 'filestore-xfs', 'kernels/pre-single-major', 'krbd/unmap/{ceph/ceph', 'tasks/unmap}'] 2022-02-03T14:20:27.690 INFO:scrape: 2022-02-03T14:20:27.691 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rbd/concurrent.sh 2022-02-03T14:20:27.691 INFO:scrape:1 jobs: ['6658412'] 2022-02-03T14:20:27.692 INFO:scrape:suites: ['clusters/fixed-3', 'conf', 'install/ceph', 'krbd/rbd-nomount/{bluestore-bitmap', 'ms_mode/crc$/{crc}', 'msgr-failures/many', 'tasks/rbd_concurrent}'] 2022-02-03T14:20:27.695 INFO:scrape: 2022-02-03T14:20:27.699 INFO:teuthology.results:Sending results to idryomov@gmail.com: Test Run: dis-2022-02-03_10:26:38-krbd-wip-krbd-rxbounce-option-testing-default-smithi ================================================================= info: http://pulpito.ceph.com/dis-2022-02-03_10:26:38-krbd-wip-krbd-rxbounce-option-testing-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/dis-2022-02-03_10:26:38-krbd-wip-krbd-rxbounce-option-testing-default-smithi/ failed: 4 dead: 0 running: 0 waiting: 0 queued: 0 passed: 2 Fail ================================================================= [6658410] krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/pre-single-major tasks/unmap} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/dis-2022-02-03_10:26:38-krbd-wip-krbd-rxbounce-option-testing-default-smithi/6658410/ log: http://qa-proxy.ceph.com/teuthology/dis-2022-02-03_10:26:38-krbd-wip-krbd-rxbounce-option-testing-default-smithi/6658410/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3c5d3add6ce14d0fbaa232ba5d8a2a96 Failed to fetch package version from https://shaman.ceph.com/api/search/?st atus=ready&project=kernel&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&re f=nightly_pre-single-major [6658412] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_concurrent} ----------------------------------------------------------------- time: 03:39:34 info: http://pulpito.ceph.com/dis-2022-02-03_10:26:38-krbd-wip-krbd-rxbounce-option-testing-default-smithi/6658412/ log: http://qa-proxy.ceph.com/teuthology/dis-2022-02-03_10:26:38-krbd-wip-krbd-rxbounce-option-testing-default-smithi/6658412/ sentry: https://sentry.ceph.com/organizations/ceph/?query=df9dcf6111974cf5998d170fd81faf1b Command failed (workunit test rbd/concurrent.sh) on smithi139 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=239f3b058ecd10b05894b75aab82b58c444cda60 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/concurrent.sh' [6658414] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/legacy$/{legacy-rxbounce} msgr-failures/many tasks/krbd_latest_osdmap_on_map} ----------------------------------------------------------------- time: 00:19:27 info: http://pulpito.ceph.com/dis-2022-02-03_10:26:38-krbd-wip-krbd-rxbounce-option-testing-default-smithi/6658414/ log: http://qa-proxy.ceph.com/teuthology/dis-2022-02-03_10:26:38-krbd-wip-krbd-rxbounce-option-testing-default-smithi/6658414/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3fbe246e2b8145d4bd9d0be2efbdabb1 "2022-02-03T10:59:06.239245+0000 mon.a (mon.0) 229 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6658415] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/crc$/{crc} msgr-failures/many tasks/rbd_image_read} ----------------------------------------------------------------- time: 00:19:12 info: http://pulpito.ceph.com/dis-2022-02-03_10:26:38-krbd-wip-krbd-rxbounce-option-testing-default-smithi/6658415/ log: http://qa-proxy.ceph.com/teuthology/dis-2022-02-03_10:26:38-krbd-wip-krbd-rxbounce-option-testing-default-smithi/6658415/ sentry: https://sentry.ceph.com/organizations/ceph/?query=75dbc466a356443f83f45f891d397e2f "2022-02-03T10:59:20.322665+0000 mon.a (mon.0) 308 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log Pass ================================================================= [6658411] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/krbd_udev_enumerate} time: 00:38:41 info: http://pulpito.ceph.com/dis-2022-02-03_10:26:38-krbd-wip-krbd-rxbounce-option-testing-default-smithi/6658411/ [6658413] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph ms_mode/secure msgr-failures/many tasks/rbd_image_read} time: 00:11:09 info: http://pulpito.ceph.com/dis-2022-02-03_10:26:38-krbd-wip-krbd-rxbounce-option-testing-default-smithi/6658413/