2022-01-28T22:47:51.970 INFO:root:teuthology version: 1.1.0-7c0cb867 2022-01-28T22:47:51.972 INFO:teuthology.results:subset: '12/99' 2022-01-28T22:47:51.972 INFO:teuthology.results:seed: '2053' 2022-01-28T22:59:09.008 INFO:root:teuthology version: 1.1.0-7c0cb867 2022-01-28T22:59:09.013 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-01-29T05:34:38.140 INFO:teuthology.results:Tests finished! gathering results... 2022-01-29T05:34:39.549 INFO:scrape:Found 14 jobs 2022-01-29T05:34:39.941 INFO:scrape:Found 3 distinct failure reasons 2022-01-29T05:34:39.942 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rbd/qemu_dynamic_features.sh 2022-01-29T05:34:39.943 INFO:scrape:2 jobs: ['6648115', '6648105'] 2022-01-29T05:34:39.943 INFO:scrape:suites intersection: ['clusters/{fixed-3', 'openstack}', 'qemu/xfstests', 'rbd/maintenance/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'workloads/dynamic_features_no_cache}'] 2022-01-29T05:34:39.944 INFO:scrape:suites union: ['clusters/{fixed-3', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-comp-zstd', 'openstack}', 'qemu/xfstests', 'rbd/maintenance/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'workloads/dynamic_features_no_cache}'] 2022-01-29T05:34:39.944 INFO:scrape: 2022-01-29T05:34:39.945 INFO:scrape:Failure: reached maximum tries (90) after waiting for 540 seconds 2022-01-29T05:34:39.945 INFO:scrape:1 jobs: ['6648114'] 2022-01-29T05:34:39.946 INFO:scrape:suites: ['clusters/{fixed-3', 'config/permit-partial-discard', 'min-compat-client/default', 'msgr-failures/few', 'objectstore/bluestore-hybrid', 'openstack}', 'pool/replicated-data-pool', 'rbd/librbd/{cache/writethrough', 'supported-random-distro$/{centos_8.stream}', 'workloads/python_api_tests_with_journaling}'] 2022-01-29T05:34:39.946 INFO:scrape: 2022-01-29T05:34:39.947 INFO:scrape:Dead: b'2022-01-29T05:29:19.881 DEBUG:teuthology.exit:Finished running handlers' 2022-01-29T05:34:39.947 INFO:scrape:1 jobs: ['6648112'] 2022-01-29T05:34:39.948 INFO:scrape:suites: ['2-cluster/{fix-2', '3-supported-random-distro$/{rhel_8}', '4-pool/big-cache', '5-cache-mode/rwl', '6-workloads/qemu_xfstests}', 'openstack}', 'rbd/persistent-writeback-cache/{1-base/install'] 2022-01-29T05:34:39.948 INFO:scrape: 2022-01-29T05:34:39.949 INFO:teuthology.results:Sending results to idryomov@gmail.com: Test Run: dis-2022-01-28_22:47:14-rbd-wip-rbd-quicy-batch-1-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/dis-2022-01-28_22:47:14-rbd-wip-rbd-quicy-batch-1-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/dis-2022-01-28_22:47:14-rbd-wip-rbd-quicy-batch-1-distro-default-smithi/ failed: 3 dead: 1 running: 0 waiting: 0 queued: 0 passed: 10 Fail ================================================================= [6648105] rbd/maintenance/{base/install clusters/{fixed-3 openstack} objectstore/bluestore-comp-zstd qemu/xfstests supported-random-distro$/{ubuntu_latest} workloads/dynamic_features_no_cache} ----------------------------------------------------------------- time: 03:14:02 info: http://pulpito.ceph.com/dis-2022-01-28_22:47:14-rbd-wip-rbd-quicy-batch-1-distro-default-smithi/6648105/ log: http://qa-proxy.ceph.com/teuthology/dis-2022-01-28_22:47:14-rbd-wip-rbd-quicy-batch-1-distro-default-smithi/6648105/ sentry: https://sentry.ceph.com/organizations/ceph/?query=21b3f0f57f2c4cee8b9ae8bb2f6158ed Command failed (workunit test rbd/qemu_dynamic_features.sh) on smithi116 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=90d351aefbb203c9b6a5d243aca4b350b7b71003 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 IMAGE_NAME=client.0.1-clone adjust- ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /ho me/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/qemu_dynamic_features.sh ' [6648114] rbd/librbd/{cache/writethrough clusters/{fixed-3 openstack} config/permit-partial-discard min-compat-client/default msgr-failures/few objectstore/bluestore-hybrid pool/replicated-data-pool supported-random-distro$/{centos_8.stream} workloads/python_api_tests_with_journaling} ----------------------------------------------------------------- time: 00:19:14 info: http://pulpito.ceph.com/dis-2022-01-28_22:47:14-rbd-wip-rbd-quicy-batch-1-distro-default-smithi/6648114/ log: http://qa-proxy.ceph.com/teuthology/dis-2022-01-28_22:47:14-rbd-wip-rbd-quicy-batch-1-distro-default-smithi/6648114/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4953970493e94fc4a57344a9e8108fae reached maximum tries (90) after waiting for 540 seconds [6648115] rbd/maintenance/{base/install clusters/{fixed-3 openstack} objectstore/bluestore-comp-snappy qemu/xfstests supported-random-distro$/{ubuntu_latest} workloads/dynamic_features_no_cache} ----------------------------------------------------------------- time: 03:14:02 info: http://pulpito.ceph.com/dis-2022-01-28_22:47:14-rbd-wip-rbd-quicy-batch-1-distro-default-smithi/6648115/ log: http://qa-proxy.ceph.com/teuthology/dis-2022-01-28_22:47:14-rbd-wip-rbd-quicy-batch-1-distro-default-smithi/6648115/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d6fa79144ba14520904c9a6f214be046 Command failed (workunit test rbd/qemu_dynamic_features.sh) on smithi204 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=90d351aefbb203c9b6a5d243aca4b350b7b71003 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 IMAGE_NAME=client.0.1-clone adjust- ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /ho me/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/qemu_dynamic_features.sh ' Dead ================================================================= [6648112] rbd/persistent-writeback-cache/{1-base/install 2-cluster/{fix-2 openstack} 3-supported-random-distro$/{rhel_8} 4-pool/big-cache 5-cache-mode/rwl 6-workloads/qemu_xfstests} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/dis-2022-01-28_22:47:14-rbd-wip-rbd-quicy-batch-1-distro-default-smithi/6648112/ log: http://qa-proxy.ceph.com/teuthology/dis-2022-01-28_22:47:14-rbd-wip-rbd-quicy-batch-1-distro-default-smithi/6648112/ hit max job timeout Pass ================================================================= [6648106] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8.stream} workloads/rbd-mirror-fsx-workunit} time: 00:36:36 info: http://pulpito.ceph.com/dis-2022-01-28_22:47:14-rbd-wip-rbd-quicy-batch-1-distro-default-smithi/6648106/ [6648107] rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_nbd} time: 00:09:43 info: http://pulpito.ceph.com/dis-2022-01-28_22:47:14-rbd-wip-rbd-quicy-batch-1-distro-default-smithi/6648107/ [6648108] rbd/librbd/{cache/writeback clusters/{fixed-3 openstack} config/none min-compat-client/default msgr-failures/few objectstore/bluestore-comp-lz4 pool/none supported-random-distro$/{rhel_8} workloads/c_api_tests} time: 00:36:17 info: http://pulpito.ceph.com/dis-2022-01-28_22:47:14-rbd-wip-rbd-quicy-batch-1-distro-default-smithi/6648108/ [6648109] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-minimum} time: 00:23:23 info: http://pulpito.ceph.com/dis-2022-01-28_22:47:14-rbd-wip-rbd-quicy-batch-1-distro-default-smithi/6648109/ [6648110] rbd/qemu/{cache/writearound clusters/{fixed-3 openstack} features/defaults msgr-failures/few objectstore/bluestore-comp-zlib pool/ec-data-pool supported-random-distro$/{rhel_8} workloads/qemu_xfstests} time: 02:19:12 info: http://pulpito.ceph.com/dis-2022-01-28_22:47:14-rbd-wip-rbd-quicy-batch-1-distro-default-smithi/6648110/ [6648111] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/filestore-xfs validator/memcheck workloads/python_api_tests_with_journaling} time: 00:32:31 info: http://pulpito.ceph.com/dis-2022-01-28_22:47:14-rbd-wip-rbd-quicy-batch-1-distro-default-smithi/6648111/ [6648113] rbd/librbd/{cache/writethrough clusters/{fixed-3 openstack} config/none min-compat-client/default msgr-failures/few objectstore/bluestore-comp-lz4 pool/replicated-data-pool supported-random-distro$/{ubuntu_latest} workloads/c_api_tests_with_journaling} time: 00:35:22 info: http://pulpito.ceph.com/dis-2022-01-28_22:47:14-rbd-wip-rbd-quicy-batch-1-distro-default-smithi/6648113/ [6648116] rbd/librbd/{cache/writeback clusters/{fixed-3 openstack} config/none min-compat-client/octopus msgr-failures/few objectstore/bluestore-stupid pool/none supported-random-distro$/{ubuntu_latest} workloads/c_api_tests_with_defaults} time: 00:25:31 info: http://pulpito.ceph.com/dis-2022-01-28_22:47:14-rbd-wip-rbd-quicy-batch-1-distro-default-smithi/6648116/ [6648117] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-fsx-workunit} time: 00:34:36 info: http://pulpito.ceph.com/dis-2022-01-28_22:47:14-rbd-wip-rbd-quicy-batch-1-distro-default-smithi/6648117/ [6648118] rbd/cli_v1/{base/install clusters/{fixed-1 openstack} features/format-1 msgr-failures/few objectstore/bluestore-comp-lz4 pool/none supported-random-distro$/{ubuntu_latest} workloads/rbd_cli_generic} time: 00:19:18 info: http://pulpito.ceph.com/dis-2022-01-28_22:47:14-rbd-wip-rbd-quicy-batch-1-distro-default-smithi/6648118/