2020-11-18T15:24:54.148 INFO:root:teuthology version: 1.0.0-357fd4a 2020-11-18T15:24:54.149 INFO:teuthology.results:seed: '5168' 2020-11-18T15:34:19.049 INFO:root:teuthology version: 1.0.0-357fd4a 2020-11-18T15:34:19.083 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-11-18T16:49:20.401 INFO:teuthology.results:Tests finished! gathering results... 2020-11-18T16:49:21.605 INFO:scrape:Found 12 jobs 2020-11-18T16:49:23.474 INFO:scrape:Found 1 distinct failure reasons 2020-11-18T16:49:23.475 INFO:scrape:Failure: Command failed on smithi099 with status 160: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 20000 -K -U pool_client.0 image_client.0' 2020-11-18T16:49:23.475 INFO:scrape:12 jobs: ['5636230', '5636236', '5636238', '5636235', '5636234', '5636237', '5636241', '5636231', '5636232', '5636233', '5636240', '5636239'] 2020-11-18T16:49:23.476 INFO:scrape:suites intersection: ['clusters/3-node', 'conf', 'krbd:fsx/{ceph/ceph'] 2020-11-18T16:49:23.476 INFO:scrape:suites union: ['clusters/3-node', 'conf', 'krbd:fsx/{ceph/ceph', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'randomized-striping-off}', 'randomized-striping-on}', 'striping/default/{msgr-failures/few', 'striping/default/{msgr-failures/many', 'striping/fancy/{msgr-failures/few', 'tasks/fsx-1-client}', 'tasks/fsx-3-client}'] 2020-11-18T16:49:23.477 INFO:scrape: 2020-11-18T16:49:23.478 INFO:teuthology.results:Sending results to idryomov@gmail.com: Test Run: dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/ failed: 12 dead: 0 running: 0 waiting: 0 queued: 0 passed: 0 Fail ================================================================= [5636230] krbd:fsx/{ceph/ceph clusters/3-node conf objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} ----------------------------------------------------------------- time: 00:06:47 info: http://pulpito.ceph.com/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/5636230/ log: http://qa-proxy.ceph.com/teuthology/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/5636230/ sentry: https://sentry.ceph.com/organizations/ceph/?query=69aee4adf1f64041bbb61931ec89f9e9 Command failed on smithi099 with status 160: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 20000 -K -U pool_client.0 image_client.0' [5636231] krbd:fsx/{ceph/ceph clusters/3-node conf objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} ----------------------------------------------------------------- time: 00:06:57 info: http://pulpito.ceph.com/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/5636231/ log: http://qa-proxy.ceph.com/teuthology/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/5636231/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bbe801dc67e847998b5e40636e026a7c Command failed on smithi118 with status 150: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 1 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 10000 -K pool_client.1 image_client.1' [5636232] krbd:fsx/{ceph/ceph clusters/3-node conf objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} ----------------------------------------------------------------- time: 00:06:35 info: http://pulpito.ceph.com/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/5636232/ log: http://qa-proxy.ceph.com/teuthology/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/5636232/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2e2c817dbbe04de3bb7c7b3cb53374a2 Command failed on smithi138 with status 160: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 20000 -K -U pool_client.0 image_client.0' [5636233] krbd:fsx/{ceph/ceph clusters/3-node conf objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} ----------------------------------------------------------------- time: 00:06:51 info: http://pulpito.ceph.com/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/5636233/ log: http://qa-proxy.ceph.com/teuthology/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/5636233/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f0ec1e0d02434453b8c4a344b00aa6f4 Command failed on smithi145 with status 160: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 2 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 10000 -K -U pool_client.2 image_client.2' [5636234] krbd:fsx/{ceph/ceph clusters/3-node conf objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} ----------------------------------------------------------------- time: 00:06:32 info: http://pulpito.ceph.com/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/5636234/ log: http://qa-proxy.ceph.com/teuthology/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/5636234/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8c05ec9e36d544be9f8ce2530e10a8fe Command failed on smithi043 with status 160: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 20000 -K pool_client.0 image_client.0' [5636235] krbd:fsx/{ceph/ceph clusters/3-node conf objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} ----------------------------------------------------------------- time: 00:06:36 info: http://pulpito.ceph.com/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/5636235/ log: http://qa-proxy.ceph.com/teuthology/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/5636235/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4c4bc154714d49968b04eb433c651c63 Command failed on smithi152 with status 160: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 2 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 10000 -K -U pool_client.2 image_client.2' [5636236] krbd:fsx/{ceph/ceph clusters/3-node conf objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} ----------------------------------------------------------------- time: 00:06:09 info: http://pulpito.ceph.com/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/5636236/ log: http://qa-proxy.ceph.com/teuthology/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/5636236/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2cd512e46b6b41e8a7e9dbbb711c7a99 Command failed on smithi173 with status 150: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 20000 -K -U pool_client.0 image_client.0' [5636237] krbd:fsx/{ceph/ceph clusters/3-node conf objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} ----------------------------------------------------------------- time: 00:07:00 info: http://pulpito.ceph.com/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/5636237/ log: http://qa-proxy.ceph.com/teuthology/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/5636237/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9307e31836134137bd27c2bdd4d549e6 Command failed on smithi097 with status 150: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 1 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 10000 -K pool_client.1 image_client.1' [5636238] krbd:fsx/{ceph/ceph clusters/3-node conf objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} ----------------------------------------------------------------- time: 00:06:13 info: http://pulpito.ceph.com/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/5636238/ log: http://qa-proxy.ceph.com/teuthology/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/5636238/ sentry: https://sentry.ceph.com/organizations/ceph/?query=585ef7867da44218a7b13a0b32472dc2 Command failed on smithi170 with status 160: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 20000 -K -U pool_client.0 image_client.0' [5636239] krbd:fsx/{ceph/ceph clusters/3-node conf objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} ----------------------------------------------------------------- time: 00:07:10 info: http://pulpito.ceph.com/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/5636239/ log: http://qa-proxy.ceph.com/teuthology/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/5636239/ sentry: https://sentry.ceph.com/organizations/ceph/?query=42cf30488be54a748f195061a22f45ac Command failed on smithi110 with status 160: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 10000 -K -U pool_client.0 image_client.0' [5636240] krbd:fsx/{ceph/ceph clusters/3-node conf objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} ----------------------------------------------------------------- time: 00:05:56 info: http://pulpito.ceph.com/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/5636240/ log: http://qa-proxy.ceph.com/teuthology/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/5636240/ sentry: https://sentry.ceph.com/organizations/ceph/?query=11eb0cea943d428da6ae997dd243d987 Command failed on smithi012 with status 160: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 20000 -K pool_client.0 image_client.0' [5636241] krbd:fsx/{ceph/ceph clusters/3-node conf objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} ----------------------------------------------------------------- time: 00:06:57 info: http://pulpito.ceph.com/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/5636241/ log: http://qa-proxy.ceph.com/teuthology/dis-2020-11-18_15:24:40-krbd:fsx-nautilus-distro-basic-smithi/5636241/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d6754e35de2d491e9faa3d36d66fffe3 Command failed on smithi031 with status 165: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 2 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 10000 -K -U pool_client.2 image_client.2'