2021-08-23T15:52:32.425 INFO:root:teuthology version: 1.1.0-321319b1 2021-08-23T15:52:32.426 INFO:teuthology.results:subset: '10486/100000' 2021-08-23T15:52:32.427 INFO:teuthology.results:seed: '7647' 2021-08-24T12:36:07.543 INFO:root:teuthology version: 1.1.0-321319b1 2021-08-24T12:36:07.584 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-08-25T00:36:04.422 INFO:teuthology.results:Tests finished! gathering results... 2021-08-25T00:36:22.312 INFO:scrape:Found 180 jobs 2021-08-25T00:36:28.141 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351824/teuthology.log 2021-08-25T00:36:28.142 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351824/teuthology.log 2021-08-25T00:36:29.402 INFO:scrape:Found 11 distinct failure reasons 2021-08-25T00:36:29.403 INFO:scrape:Dead: b'2021-08-25T00:33:04.365 DEBUG:teuthology.exit:Finished running handlers' 2021-08-25T00:36:29.403 INFO:scrape:ceph version 17.0.0-7140-g8aaa6b0f (8aaa6b0fef0256e42fe006a18b61f7f74db7106c) quincy (dev) 1: /lib64/libpthread.so.0(+0x12b20) [0x7f9e2d91bb20] 2: gsignal() 3: abort() 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x1b8) [0x558ef82e3290] 5: (KernelDevice::_aio_thread()+0xf9e) [0x558ef8ef5c6e] 6: (KernelDevice::AioCompletionThread::entry()+0x11) [0x558ef8f01a11] 7: (Thread::_entry_func(void*)+0xd) [0x558ef8acd61d] 8: /lib64/libpthread.so.0(+0x814a) [0x7f9e2d91114a] 9: clone() 2021-08-25T00:36:29.404 INFO:scrape:1 jobs: ['6351943'] 2021-08-25T00:36:29.404 INFO:scrape:suites: ['clients/mirror', 'cluster/{2-node', 'msgr-failures/few', 'objectstore/bluestore-hybrid', 'openstack}', 'policy/simple', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'supported-random-distro$/{centos_8.stream}', 'workloads/rbd-mirror-snapshot-stress-workunit-minimum}'] 2021-08-25T00:36:29.405 INFO:scrape: 2021-08-25T00:36:29.405 INFO:scrape:Failure: "2021-08-24T00:12:05.189552+0000 mon.a (mon.0) 95 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log 2021-08-25T00:36:29.406 INFO:scrape:5 jobs: ['6351858', '6351798', '6351944', '6351791', '6351767'] 2021-08-25T00:36:29.406 INFO:scrape:suites intersection: ['msgr-failures/few', 'openstack}'] 2021-08-25T00:36:29.407 INFO:scrape:suites union: ['clients/mirror', 'clients/{mirror-extra', 'cluster/{2-node', 'clusters/{fixed-1', 'features/defaults', 'mirror}', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-hybrid', 'objectstore/filestore-xfs', 'openstack}', 'policy/none', 'policy/simple', 'pool/none', 'rbd-mirror/four-per-cluster', 'rbd/cli/{base/install', 'rbd/mirror-thrash/{base/install', 'rbd/mirror/{base/install', 'supported-random-distro$/{centos_8.stream}', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rbd-mirror-fsx-workunit}', 'workloads/rbd-mirror-snapshot-workunit-exclusive-lock}', 'workloads/rbd-mirror-snapshot-workunit-fast-diff}', 'workloads/rbd_cli_generic}'] 2021-08-25T00:36:29.408 INFO:scrape: 2021-08-25T00:36:29.408 INFO:scrape:Crash: "2021-08-23T18:36:44.408633+0000 mon.a (mon.0) 525 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log 2021-08-25T00:36:29.409 INFO:scrape:ceph version 17.0.0-7140-g8aaa6b0f (8aaa6b0fef0256e42fe006a18b61f7f74db7106c) quincy (dev) 1: /lib64/libpthread.so.0(+0x12b20) [0x7f9363479b20] 2: gsignal() 3: abort() 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x1b8) [0x55630a080290] 5: (KernelDevice::_aio_thread()+0xf9e) [0x55630ac92c6e] 6: (KernelDevice::AioCompletionThread::entry()+0x11) [0x55630ac9ea11] 7: (Thread::_entry_func(void*)+0xd) [0x55630a86a61d] 8: /lib64/libpthread.so.0(+0x814a) [0x7f936346f14a] 9: clone() 2021-08-25T00:36:29.409 INFO:scrape:1 jobs: ['6351782'] 2021-08-25T00:36:29.410 INFO:scrape:suites: ['clusters/{fixed-3', 'features/defaults', 'msgr-failures/few', 'objectstore/bluestore-stupid', 'openstack}', 'pool/small-cache-pool', 'rbd/encryption/{cache/none', 'supported-random-distro$/{centos_8}', 'workloads/qemu_xfstests_luks2}'] 2021-08-25T00:36:29.410 INFO:scrape: 2021-08-25T00:36:29.411 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rbd/qemu_rebuild_object_map.sh 2021-08-25T00:36:29.411 INFO:scrape:1 jobs: ['6351830'] 2021-08-25T00:36:29.412 INFO:scrape:suites: ['clusters/{fixed-3', 'objectstore/bluestore-stupid', 'openstack}', 'qemu/xfstests', 'rbd/maintenance/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rebuild_object_map}'] 2021-08-25T00:36:29.412 INFO:scrape: 2021-08-25T00:36:29.413 INFO:scrape:Failure: Command failed (workunit test rbd/test_librbd.sh) on gibba036 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=63ae3147b2c6f637c8316e6db63e21cf060d4914 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=61 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd.sh' 2021-08-25T00:36:29.413 INFO:scrape:2 jobs: ['6351859', '6351768'] 2021-08-25T00:36:29.414 INFO:scrape:suites intersection: ['2-cluster/{fix-2', '4-pool/cache', '6-workloads/c_api_tests_with_defaults}', 'openstack}', 'rbd/persistent-writeback-cache/{1-base/install'] 2021-08-25T00:36:29.414 INFO:scrape:suites union: ['2-cluster/{fix-2', '3-supported-random-distro$/{centos_8.stream}', '3-supported-random-distro$/{rhel_8}', '4-pool/cache', '5-cache-mode/rwl', '5-cache-mode/ssd', '6-workloads/c_api_tests_with_defaults}', 'openstack}', 'rbd/persistent-writeback-cache/{1-base/install'] 2021-08-25T00:36:29.415 INFO:scrape: 2021-08-25T00:36:29.415 INFO:scrape:Failure: Command failed on gibba041 with status 151: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0' 2021-08-25T00:36:29.415 INFO:scrape:2 jobs: ['6351910', '6351940'] 2021-08-25T00:36:29.416 INFO:scrape:suites intersection: ['cluster/{fixed-3', 'msgr-failures/few', 'openstack}', 'rbd/nbd/{base/install', 'thrashers/cache', 'thrashosds-health', 'workloads/rbd_fsx_nbd}'] 2021-08-25T00:36:29.416 INFO:scrape:suites union: ['cluster/{fixed-3', 'msgr-failures/few', 'objectstore/bluestore-low-osd-mem-target', 'objectstore/filestore-xfs', 'openstack}', 'rbd/nbd/{base/install', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/cache', 'thrashosds-health', 'workloads/rbd_fsx_nbd}'] 2021-08-25T00:36:29.417 INFO:scrape: 2021-08-25T00:36:29.418 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rbd/cli_generic.sh 2021-08-25T00:36:29.418 INFO:scrape:1 jobs: ['6351786'] 2021-08-25T00:36:29.419 INFO:scrape:suites: ['clusters/{fixed-1', 'features/format-1', 'msgr-failures/few', 'objectstore/bluestore-stupid', 'openstack}', 'pool/small-cache-pool', 'rbd/cli_v1/{base/install', 'supported-random-distro$/{centos_8}', 'workloads/rbd_cli_generic}'] 2021-08-25T00:36:29.419 INFO:scrape: 2021-08-25T00:36:29.420 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rbd/qemu_dynamic_features.sh 2021-08-25T00:36:29.420 INFO:scrape:1 jobs: ['6351818'] 2021-08-25T00:36:29.420 INFO:scrape:suites: ['clusters/{fixed-3', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'qemu/xfstests', 'rbd/maintenance/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'workloads/dynamic_features_no_cache}'] 2021-08-25T00:36:29.421 INFO:scrape: 2021-08-25T00:36:29.421 INFO:scrape:Failure: reached maximum tries (90) after waiting for 540 seconds 2021-08-25T00:36:29.422 INFO:scrape:1 jobs: ['6351840'] 2021-08-25T00:36:29.422 INFO:scrape:suites: ['clients/mirror', 'cluster/{2-node', 'msgr-failures/few', 'objectstore/filestore-xfs', 'openstack}', 'policy/none', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}'] 2021-08-25T00:36:29.423 INFO:scrape: 2021-08-25T00:36:29.423 INFO:scrape:Failure: Command failed on gibba027 with status 1: 'test -f /home/ubuntu/cephtest/archive/qemu/client.0/success' 2021-08-25T00:36:29.424 INFO:scrape:1 jobs: ['6351814'] 2021-08-25T00:36:29.424 INFO:scrape:suites: ['2-cluster/{fix-2', '3-supported-random-distro$/{centos_8.stream}', '4-pool/cache', '5-cache-mode/ssd', '6-workloads/qemu_xfstests}', 'openstack}', 'rbd/persistent-writeback-cache/{1-base/install'] 2021-08-25T00:36:29.424 INFO:scrape: 2021-08-25T00:36:29.425 INFO:scrape:Dead: None 2021-08-25T00:36:29.425 INFO:scrape:1 jobs: ['6351824'] 2021-08-25T00:36:29.426 INFO:scrape: 2021-08-25T00:36:29.432 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba ================================================================= info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/ failed: 15 dead: 2 running: 0 waiting: 0 queued: 0 passed: 163 Fail ================================================================= [6351767] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_8.stream} workloads/rbd-mirror-snapshot-workunit-exclusive-lock} ----------------------------------------------------------------- time: 00:16:34 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351767/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351767/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ccf9fe2e9f3742e8bdca9ba7b4b955f2 "2021-08-23T16:18:48.184425+0000 mon.a (mon.0) 89 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6351768] rbd/persistent-writeback-cache/{1-base/install 2-cluster/{fix-2 openstack} 3-supported-random-distro$/{rhel_8} 4-pool/cache 5-cache-mode/ssd 6-workloads/c_api_tests_with_defaults} ----------------------------------------------------------------- time: 00:31:46 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351768/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351768/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2c7df34a3c504299882d7f39a662fc9c Command failed (workunit test rbd/test_librbd.sh) on gibba028 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=63ae3147b2c6f637c8316e6db63e21cf060d4914 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=61 adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd.sh' [6351782] rbd/encryption/{cache/none clusters/{fixed-3 openstack} features/defaults msgr-failures/few objectstore/bluestore-stupid pool/small-cache-pool supported-random-distro$/{centos_8} workloads/qemu_xfstests_luks2} ----------------------------------------------------------------- time: 02:01:33 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351782/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351782/ "2021-08-23T18:36:44.408633+0000 mon.a (mon.0) 525 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log [6351786] rbd/cli_v1/{base/install clusters/{fixed-1 openstack} features/format-1 msgr-failures/few objectstore/bluestore-stupid pool/small-cache-pool supported-random-distro$/{centos_8} workloads/rbd_cli_generic} ----------------------------------------------------------------- time: 03:31:41 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351786/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351786/ sentry: https://sentry.ceph.com/organizations/ceph/?query=dc609db75c5b40b3905fd6434c71d4ea Command failed (workunit test rbd/cli_generic.sh) on gibba022 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=63ae3147b2c6f637c8316e6db63e21cf060d4914 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/cli_generic.sh' [6351791] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8.stream} workloads/rbd-mirror-fsx-workunit} ----------------------------------------------------------------- time: 00:34:11 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351791/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351791/ "2021-08-23T17:34:57.385590+0000 mon.a (mon.0) 214 : cluster [WRN] Health check failed: Degraded data redundancy: 4/8746 objects degraded (0.046%), 1 pg degraded (PG_DEGRADED)" in cluster log [6351798] rbd/cli/{base/install clusters/{fixed-1 openstack} features/defaults msgr-failures/few objectstore/filestore-xfs pool/none supported-random-distro$/{centos_8} workloads/rbd_cli_generic} ----------------------------------------------------------------- time: 00:15:41 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351798/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351798/ sentry: https://sentry.ceph.com/organizations/ceph/?query=29c14d88123a4c54b241d80d238cecdd "2021-08-23T18:10:29.533910+0000 mon.a (mon.0) 108 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6351814] rbd/persistent-writeback-cache/{1-base/install 2-cluster/{fix-2 openstack} 3-supported-random-distro$/{centos_8.stream} 4-pool/cache 5-cache-mode/ssd 6-workloads/qemu_xfstests} ----------------------------------------------------------------- time: 00:18:09 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351814/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351814/ Command failed on gibba027 with status 1: 'test -f /home/ubuntu/cephtest/archive/qemu/client.0/success' [6351818] rbd/maintenance/{base/install clusters/{fixed-3 openstack} objectstore/bluestore-low-osd-mem-target qemu/xfstests supported-random-distro$/{ubuntu_latest} workloads/dynamic_features_no_cache} ----------------------------------------------------------------- time: 03:10:24 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351818/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351818/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9d7aef5ee79944019655bce4293cb3a1 Command failed (workunit test rbd/qemu_dynamic_features.sh) on gibba045 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=63ae3147b2c6f637c8316e6db63e21cf060d4914 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 IMAGE_NAME=client.0.1-clone adjust- ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /ho me/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/qemu_dynamic_features.sh ' [6351830] rbd/maintenance/{base/install clusters/{fixed-3 openstack} objectstore/bluestore-stupid qemu/xfstests supported-random-distro$/{ubuntu_latest} workloads/rebuild_object_map} ----------------------------------------------------------------- time: 03:12:26 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351830/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351830/ sentry: https://sentry.ceph.com/organizations/ceph/?query=574803d2513d4367b81d9d3fb750f931 Command failed (workunit test rbd/qemu_rebuild_object_map.sh) on gibba037 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=63ae3147b2c6f637c8316e6db63e21cf060d4914 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 IMAGE_NAME=client.0.1-clone adjust- ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /ho me/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/qemu_rebuild_object_map. sh' [6351840] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} ----------------------------------------------------------------- time: 00:14:10 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351840/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351840/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5f592a27bbd7419b9e6ba71111cbcd39 reached maximum tries (90) after waiting for 540 seconds [6351858] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-fsx-workunit} ----------------------------------------------------------------- time: 00:14:40 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351858/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351858/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8525fb85e8f84e42ba512cbf6637ff61 "2021-08-24T00:12:05.189552+0000 mon.a (mon.0) 95 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6351859] rbd/persistent-writeback-cache/{1-base/install 2-cluster/{fix-2 openstack} 3-supported-random-distro$/{centos_8.stream} 4-pool/cache 5-cache-mode/rwl 6-workloads/c_api_tests_with_defaults} ----------------------------------------------------------------- time: 00:38:44 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351859/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351859/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f188de6662174e5990054832ccde2071 Command failed (workunit test rbd/test_librbd.sh) on gibba036 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=63ae3147b2c6f637c8316e6db63e21cf060d4914 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=61 adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd.sh' [6351910] rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8} thrashers/cache thrashosds-health workloads/rbd_fsx_nbd} ----------------------------------------------------------------- time: 00:17:04 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351910/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351910/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0106503a438847c9b78fa3eacf9ee61e Command failed on gibba041 with status 151: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0' [6351940] rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{ubuntu_latest} thrashers/cache thrashosds-health workloads/rbd_fsx_nbd} ----------------------------------------------------------------- time: 00:14:49 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351940/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351940/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5877298b71bb433291e02e3a7655e5a7 Command failed on gibba042 with status 160: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0' [6351944] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{centos_8.stream} workloads/rbd-mirror-snapshot-workunit-fast-diff} ----------------------------------------------------------------- time: 00:17:41 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351944/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351944/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3f88b235ee744a2a9390c170f89337ca "2021-08-24T12:44:06.697539+0000 mon.a (mon.0) 112 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log Dead ================================================================= [6351824] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-stress-workunit-min-compat-client-octopus} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351824/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351824/ Error reimaging machines: reached maximum tries (100) after waiting for 600 seconds [6351943] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8.stream} workloads/rbd-mirror-snapshot-stress-workunit-minimum} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351943/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351943/ hit max job timeout Pass ================================================================= [6351769] rbd/singleton-bluestore/{all/issue-20295 objectstore/bluestore-comp-snappy openstack supported-random-distro$/{centos_8}} time: 00:18:49 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351769/ [6351770] rbd/migration/{1-base/install 2-clusters/{fixed-3 openstack} 3-objectstore/bluestore-comp-snappy 4-supported-random-distro$/{ubuntu_latest} 5-pool/ec-data-pool 6-prepare/raw-file 7-io-workloads/qemu_xfstests 8-migrate-workloads/execute 9-cleanup/cleanup} time: 02:30:06 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351770/ [6351771] rbd/singleton/{all/qemu-iotests-no-cache objectstore/bluestore-stupid openstack supported-random-distro$/{rhel_8}} time: 00:15:13 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351771/ [6351772] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8.stream} workloads/rbd-mirror-snapshot-stress-workunit-minimum} time: 00:46:44 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351772/ [6351773] rbd/thrash/{base/install clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/rbd_api_tests_journaling} time: 00:42:00 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351773/ [6351774] rbd/cli_v1/{base/install clusters/{fixed-1 openstack} features/format-1 msgr-failures/few objectstore/bluestore-low-osd-mem-target pool/none supported-random-distro$/{centos_8.stream} workloads/rbd_cli_import_export} time: 00:08:17 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351774/ [6351775] rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8} thrashers/cache thrashosds-health workloads/rbd_nbd} time: 00:09:43 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351775/ [6351776] rbd/librbd/{cache/writeback clusters/{fixed-3 openstack} config/none min-compat-client/default msgr-failures/few objectstore/filestore-xfs pool/none supported-random-distro$/{centos_8.stream} workloads/c_api_tests_with_journaling} time: 00:25:24 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351776/ [6351777] rbd/basic/{base/install cachepool/none clusters/{fixed-1 openstack} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8} tasks/rbd_cls_tests} time: 00:08:52 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351777/ [6351778] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-comp-zstd validator/memcheck workloads/c_api_tests_with_defaults} time: 00:38:49 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351778/ [6351779] rbd/qemu/{cache/writearound clusters/{fixed-3 openstack} features/journaling msgr-failures/few objectstore/bluestore-comp-zlib pool/none supported-random-distro$/{centos_8.stream} workloads/qemu_bonnie} time: 00:22:29 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351779/ [6351780] rbd/maintenance/{base/install clusters/{fixed-3 openstack} objectstore/bluestore-comp-zlib qemu/xfstests supported-random-distro$/{centos_8.stream} workloads/dynamic_features_no_cache} time: 02:02:44 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351780/ [6351781] rbd/cli/{base/install clusters/{fixed-1 openstack} features/layering msgr-failures/few objectstore/bluestore-stupid pool/ec-data-pool supported-random-distro$/{centos_8.stream} workloads/rbd_cli_migration} time: 00:24:32 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351781/ [6351783] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-fast-diff} time: 00:15:31 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351783/ [6351784] rbd/singleton/{all/qemu-iotests-writearound objectstore/filestore-xfs openstack supported-random-distro$/{centos_8.stream}} time: 00:08:41 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351784/ [6351785] rbd/thrash/{base/install clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} thrashers/cache thrashosds-health workloads/rbd_api_tests_no_locking} time: 00:32:34 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351785/ [6351787] rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} time: 00:15:08 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351787/ [6351788] rbd/persistent-writeback-cache/{1-base/install 2-cluster/{fix-2 openstack} 3-supported-random-distro$/{centos_8.stream} 4-pool/cache 5-cache-mode/rwl 6-workloads/fio} time: 00:18:45 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351788/ [6351789] rbd/maintenance/{base/install clusters/{fixed-3 openstack} objectstore/bluestore-comp-zstd qemu/xfstests supported-random-distro$/{ubuntu_latest} workloads/rebuild_object_map} time: 02:40:31 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351789/ [6351790] rbd/migration/{1-base/install 2-clusters/{fixed-3 openstack} 3-objectstore/bluestore-comp-zlib 4-supported-random-distro$/{centos_8.stream} 5-pool/none 6-prepare/qcow2-file 7-io-workloads/qemu_xfstests 8-migrate-workloads/execute 9-cleanup/cleanup} time: 01:09:29 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351790/ [6351792] rbd/singleton/{all/qemu-iotests-writeback objectstore/bluestore-bitmap openstack supported-random-distro$/{ubuntu_latest}} time: 00:06:29 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351792/ [6351793] rbd/librbd/{cache/writethrough clusters/{fixed-3 openstack} config/permit-partial-discard min-compat-client/octopus msgr-failures/few objectstore/bluestore-bitmap pool/replicated-data-pool supported-random-distro$/{centos_8.stream} workloads/fsx} time: 00:15:47 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351793/ [6351794] rbd/basic/{base/install cachepool/small clusters/{fixed-1 openstack} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{rhel_8} tasks/rbd_lock_and_fence} time: 00:15:29 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351794/ [6351795] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-hybrid validator/memcheck workloads/c_api_tests_with_journaling} time: 00:43:56 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351795/ [6351796] rbd/qemu/{cache/writeback clusters/{fixed-3 openstack} features/readbalance msgr-failures/few objectstore/bluestore-comp-zstd pool/replicated-data-pool supported-random-distro$/{rhel_8} workloads/qemu_fsstress} time: 00:18:21 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351796/ [6351797] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-journaling} time: 00:15:52 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351797/ [6351799] rbd/encryption/{cache/writearound clusters/{fixed-3 openstack} features/defaults msgr-failures/few objectstore/filestore-xfs pool/ec-cache-pool supported-random-distro$/{ubuntu_latest} workloads/qemu_xfstests_luks1} time: 01:26:10 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351799/ [6351800] rbd/immutable-object-cache/{clusters/{fix-2 openstack} pool/ceph_and_immutable_object_cache supported-random-distro$/{centos_8.stream} workloads/qemu_on_immutable_object_cache_and_thrash} time: 00:53:28 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351800/ [6351801] rbd/thrash/{base/install clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound} time: 00:27:19 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351801/ [6351802] rbd/singleton/{all/qemu-iotests-writethrough objectstore/bluestore-comp-lz4 openstack supported-random-distro$/{rhel_8}} time: 00:14:28 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351802/ [6351803] rbd/maintenance/{base/install clusters/{fixed-3 openstack} objectstore/bluestore-hybrid qemu/xfstests supported-random-distro$/{centos_8} workloads/dynamic_features} time: 01:26:55 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351803/ [6351804] rbd/cli_v1/{base/install clusters/{fixed-1 openstack} features/format-1 msgr-failures/few objectstore/filestore-xfs pool/none supported-random-distro$/{ubuntu_latest} workloads/rbd_cli_import_export} time: 00:06:27 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351804/ [6351805] rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{centos_8} thrashers/cache thrashosds-health workloads/rbd_nbd} time: 00:10:55 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351805/ [6351806] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/none rbd-mirror/four-per-cluster supported-random-distro$/{rhel_8} workloads/rbd-mirror-journal-stress-workunit} time: 01:04:13 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351806/ [6351807] rbd/migration/{1-base/install 2-clusters/{fixed-3 openstack} 3-objectstore/bluestore-comp-zstd 4-supported-random-distro$/{centos_8.stream} 5-pool/replicated-data-pool 6-prepare/qcow2-http 7-io-workloads/qemu_xfstests 8-migrate-workloads/execute 9-cleanup/cleanup} time: 01:11:41 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351807/ [6351808] rbd/singleton/{all/qos objectstore/bluestore-comp-snappy openstack supported-random-distro$/{ubuntu_latest}} time: 00:25:58 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351808/ [6351809] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-minimum} time: 00:15:36 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351809/ [6351810] rbd/thrash/{base/install clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8.stream} thrashers/cache thrashosds-health workloads/rbd_fsx_cache_writeback} time: 00:12:09 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351810/ [6351811] rbd/librbd/{cache/none clusters/{fixed-3 openstack} config/copy-on-read min-compat-client/default msgr-failures/few objectstore/bluestore-comp-lz4 pool/small-cache-pool supported-random-distro$/{rhel_8} workloads/python_api_tests} time: 00:25:28 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351811/ [6351812] rbd/basic/{base/install cachepool/none clusters/{fixed-1 openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8} tasks/rbd_python_api_tests_old_format} time: 00:17:49 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351812/ [6351813] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-low-osd-mem-target validator/memcheck workloads/fsx} time: 00:15:42 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351813/ [6351815] rbd/qemu/{cache/writethrough clusters/{fixed-3 openstack} features/defaults msgr-failures/few objectstore/bluestore-hybrid pool/small-cache-pool supported-random-distro$/{centos_8} workloads/qemu_xfstests} time: 01:44:37 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351815/ [6351816] rbd/cli/{base/install clusters/{fixed-1 openstack} features/journaling msgr-failures/few objectstore/bluestore-bitmap pool/replicated-data-pool supported-random-distro$/{centos_8} workloads/rbd_cli_groups} time: 00:09:22 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351816/ [6351817] rbd/encryption/{cache/writeback clusters/{fixed-3 openstack} features/defaults msgr-failures/few objectstore/bluestore-bitmap pool/ec-data-pool supported-random-distro$/{rhel_8} workloads/qemu_xfstests_luks2} time: 01:17:51 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351817/ [6351819] rbd/singleton/{all/rbd-vs-unmanaged-snaps objectstore/bluestore-comp-zlib openstack supported-random-distro$/{ubuntu_latest}} time: 00:06:19 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351819/ [6351820] rbd/cli_v1/{base/install clusters/{fixed-1 openstack} features/format-1 msgr-failures/few objectstore/bluestore-bitmap pool/none supported-random-distro$/{centos_8} workloads/rbd_cli_generic} time: 00:18:07 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351820/ [6351821] rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} thrashers/cache thrashosds-health workloads/rbd_fsx_nbd} time: 00:13:51 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351821/ [6351822] rbd/thrash/{base/install clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writethrough} time: 00:19:25 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351822/ [6351823] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-journal-workunit} time: 00:29:58 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351823/ [6351825] rbd/migration/{1-base/install 2-clusters/{fixed-3 openstack} 3-objectstore/bluestore-hybrid 4-supported-random-distro$/{centos_8.stream} 5-pool/ec-data-pool 6-prepare/raw-file 7-io-workloads/qemu_xfstests 8-migrate-workloads/execute 9-cleanup/cleanup} time: 01:35:28 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351825/ [6351826] rbd/singleton/{all/rbd_mirror objectstore/bluestore-comp-zstd openstack supported-random-distro$/{centos_8.stream}} time: 00:41:20 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351826/ [6351827] rbd/basic/{base/install cachepool/small clusters/{fixed-1 openstack} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{centos_8} tasks/rbd_api_tests_old_format} time: 00:28:06 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351827/ [6351828] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-stupid validator/memcheck workloads/python_api_tests} time: 00:26:29 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351828/ [6351829] rbd/librbd/{cache/writearound clusters/{fixed-3 openstack} config/none min-compat-client/octopus msgr-failures/few objectstore/bluestore-comp-snappy pool/ec-data-pool supported-random-distro$/{rhel_8} workloads/python_api_tests_with_defaults} time: 00:25:54 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351829/ [6351831] rbd/qemu/{cache/none clusters/{fixed-3 openstack} features/journaling msgr-failures/few objectstore/bluestore-low-osd-mem-target pool/ec-cache-pool supported-random-distro$/{rhel_8} workloads/qemu_bonnie} time: 00:30:19 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351831/ [6351832] rbd/cli/{base/install clusters/{fixed-1 openstack} features/layering msgr-failures/few objectstore/bluestore-comp-lz4 pool/small-cache-pool supported-random-distro$/{ubuntu_latest} workloads/rbd_cli_import_export} time: 00:07:57 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351832/ [6351833] rbd/encryption/{cache/writethrough clusters/{fixed-3 openstack} features/defaults msgr-failures/few objectstore/bluestore-comp-lz4 pool/none supported-random-distro$/{rhel_8} workloads/qemu_xfstests_luks1} time: 02:26:55 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351833/ [6351834] rbd/cli_v1/{base/install clusters/{fixed-1 openstack} features/format-1 msgr-failures/few objectstore/bluestore-comp-lz4 pool/small-cache-pool supported-random-distro$/{ubuntu_latest} workloads/rbd_cli_import_export} time: 00:07:14 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351834/ [6351835] rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/rbd_nbd} time: 00:09:15 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351835/ [6351836] rbd/persistent-writeback-cache/{1-base/install 2-cluster/{fix-2 openstack} 3-supported-random-distro$/{centos_8.stream} 4-pool/cache 5-cache-mode/rwl 6-workloads/recovery} time: 00:10:56 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351836/ [6351837] rbd/singleton/{all/rbd_tasks objectstore/bluestore-hybrid openstack supported-random-distro$/{ubuntu_latest}} time: 00:06:29 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351837/ [6351838] rbd/thrash/{base/install clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{rhel_8} thrashers/cache thrashosds-health workloads/rbd_fsx_copy_on_read} time: 00:18:02 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351838/ [6351839] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-workunit-config-key} time: 00:19:53 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351839/ [6351841] rbd/maintenance/{base/install clusters/{fixed-3 openstack} objectstore/filestore-xfs qemu/xfstests supported-random-distro$/{centos_8.stream} workloads/dynamic_features} time: 01:03:43 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351841/ [6351842] rbd/migration/{1-base/install 2-clusters/{fixed-3 openstack} 3-objectstore/bluestore-low-osd-mem-target 4-supported-random-distro$/{centos_8.stream} 5-pool/none 6-prepare/qcow2-file 7-io-workloads/qemu_xfstests 8-migrate-workloads/execute 9-cleanup/cleanup} time: 01:03:11 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351842/ [6351843] rbd/singleton/{all/rbdmap_RBDMAPFILE objectstore/bluestore-low-osd-mem-target openstack supported-random-distro$/{ubuntu_latest}} time: 00:04:50 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351843/ [6351844] rbd/basic/{base/install cachepool/none clusters/{fixed-1 openstack} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{centos_8} tasks/rbd_cls_tests} time: 00:08:44 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351844/ [6351845] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/filestore-xfs validator/memcheck workloads/python_api_tests_with_defaults} time: 00:31:07 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351845/ [6351846] rbd/librbd/{cache/writeback clusters/{fixed-3 openstack} config/permit-partial-discard min-compat-client/default msgr-failures/few objectstore/bluestore-comp-zlib pool/none supported-random-distro$/{rhel_8} workloads/python_api_tests_with_journaling} time: 00:26:03 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351846/ [6351847] rbd/thrash/{base/install clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/rbd_fsx_deep_copy} time: 00:22:33 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351847/ [6351848] rbd/qemu/{cache/writearound clusters/{fixed-3 openstack} features/readbalance msgr-failures/few objectstore/bluestore-stupid pool/ec-data-pool supported-random-distro$/{centos_8.stream} workloads/qemu_fsstress} time: 00:10:51 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351848/ [6351849] rbd/cli_v1/{base/install clusters/{fixed-1 openstack} features/format-1 msgr-failures/few objectstore/bluestore-comp-snappy pool/none supported-random-distro$/{rhel_8} workloads/rbd_cli_generic} time: 00:25:20 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351849/ [6351850] rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{rhel_8} thrashers/cache thrashosds-health workloads/rbd_fsx_nbd} time: 00:24:46 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351850/ [6351851] rbd/cli/{base/install clusters/{fixed-1 openstack} features/defaults msgr-failures/few objectstore/bluestore-comp-snappy pool/ec-data-pool supported-random-distro$/{centos_8} workloads/rbd_cli_luks_encryption} time: 00:08:16 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351851/ [6351852] rbd/encryption/{cache/none clusters/{fixed-3 openstack} features/defaults msgr-failures/few objectstore/bluestore-comp-snappy pool/replicated-data-pool supported-random-distro$/{ubuntu_latest} workloads/qemu_xfstests_luks2} time: 04:01:19 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351852/ [6351853] rbd/singleton/{all/read-flags-no-cache objectstore/bluestore-stupid openstack supported-random-distro$/{centos_8}} time: 00:07:43 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351853/ [6351854] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-workunit-min-compat-client-octopus} time: 00:20:02 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351854/ [6351855] rbd/immutable-object-cache/{clusters/{fix-2 openstack} pool/ceph_and_immutable_object_cache supported-random-distro$/{rhel_8} workloads/c_api_tests_with_defaults} time: 00:29:36 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351855/ [6351856] rbd/iscsi/{base/install cluster/{fixed-3 openstack} pool/datapool supported-random-distro$/{centos_8} workloads/ceph_iscsi} time: 00:12:48 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351856/ [6351857] rbd/maintenance/{base/install clusters/{fixed-3 openstack} objectstore/bluestore-bitmap qemu/xfstests supported-random-distro$/{ubuntu_latest} workloads/dynamic_features} time: 02:38:00 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351857/ [6351860] rbd/singleton-bluestore/{all/issue-20295 objectstore/bluestore-bitmap openstack supported-random-distro$/{rhel_8}} time: 00:22:20 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351860/ [6351861] rbd/thrash/{base/install clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} thrashers/cache thrashosds-health workloads/rbd_fsx_journal} time: 00:17:10 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351861/ [6351862] rbd/migration/{1-base/install 2-clusters/{fixed-3 openstack} 3-objectstore/bluestore-stupid 4-supported-random-distro$/{ubuntu_latest} 5-pool/replicated-data-pool 6-prepare/qcow2-http 7-io-workloads/qemu_xfstests 8-migrate-workloads/execute 9-cleanup/cleanup} time: 03:10:21 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351862/ [6351863] rbd/singleton/{all/read-flags-writeback objectstore/filestore-xfs openstack supported-random-distro$/{rhel_8}} time: 00:14:17 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351863/ [6351864] rbd/basic/{base/install cachepool/small clusters/{fixed-1 openstack} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_8.stream} tasks/rbd_lock_and_fence} time: 00:08:48 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351864/ [6351865] rbd/cli_v1/{base/install clusters/{fixed-1 openstack} features/format-1 msgr-failures/few objectstore/bluestore-comp-zlib pool/small-cache-pool supported-random-distro$/{centos_8.stream} workloads/rbd_cli_import_export} time: 00:08:15 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351865/ [6351866] rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/rbd_nbd} time: 00:16:22 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351866/ [6351867] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-bitmap validator/memcheck workloads/python_api_tests_with_journaling} time: 00:27:09 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351867/ [6351868] rbd/librbd/{cache/writethrough clusters/{fixed-3 openstack} config/copy-on-read min-compat-client/octopus msgr-failures/few objectstore/bluestore-comp-zstd pool/replicated-data-pool supported-random-distro$/{rhel_8} workloads/rbd_fio} time: 00:33:57 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351868/ [6351869] rbd/qemu/{cache/writeback clusters/{fixed-3 openstack} features/defaults msgr-failures/few objectstore/filestore-xfs pool/none supported-random-distro$/{ubuntu_latest} workloads/qemu_xfstests} time: 01:18:52 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351869/ [6351870] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-workunit-policy-none} time: 00:20:27 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351870/ [6351871] rbd/cli/{base/install clusters/{fixed-1 openstack} features/journaling msgr-failures/few objectstore/bluestore-comp-zlib pool/none supported-random-distro$/{centos_8} workloads/rbd_cli_migration} time: 00:21:28 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351871/ [6351872] rbd/encryption/{cache/writearound clusters/{fixed-3 openstack} features/defaults msgr-failures/few objectstore/bluestore-comp-zlib pool/small-cache-pool supported-random-distro$/{centos_8} workloads/qemu_xfstests_luks1} time: 01:13:34 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351872/ [6351873] rbd/maintenance/{base/install clusters/{fixed-3 openstack} objectstore/bluestore-comp-lz4 qemu/xfstests supported-random-distro$/{centos_8.stream} workloads/dynamic_features_no_cache} time: 01:58:50 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351873/ [6351874] rbd/singleton/{all/read-flags-writethrough objectstore/bluestore-bitmap openstack supported-random-distro$/{rhel_8}} time: 00:14:31 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351874/ [6351875] rbd/thrash/{base/install clusters/{fixed-2 openstack} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/rbd_fsx_nocache} time: 00:30:09 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351875/ [6351876] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{rhel_8} workloads/rbd-mirror-journal-stress-workunit} time: 01:07:52 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351876/ [6351877] rbd/migration/{1-base/install 2-clusters/{fixed-3 openstack} 3-objectstore/filestore-xfs 4-supported-random-distro$/{ubuntu_latest} 5-pool/ec-data-pool 6-prepare/raw-file 7-io-workloads/qemu_xfstests 8-migrate-workloads/execute 9-cleanup/cleanup} time: 01:58:51 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351877/ [6351878] rbd/singleton/{all/snap-diff objectstore/bluestore-comp-lz4 openstack supported-random-distro$/{ubuntu_latest}} time: 00:06:46 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351878/ [6351879] rbd/cli_v1/{base/install clusters/{fixed-1 openstack} features/format-1 msgr-failures/few objectstore/bluestore-comp-zstd pool/none supported-random-distro$/{ubuntu_latest} workloads/rbd_cli_generic} time: 00:18:12 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351879/ [6351880] rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8.stream} thrashers/cache thrashosds-health workloads/rbd_fsx_nbd} time: 00:13:01 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351880/ [6351881] rbd/persistent-writeback-cache/{1-base/install 2-cluster/{fix-2 openstack} 3-supported-random-distro$/{centos_8.stream} 4-pool/cache 5-cache-mode/ssd 6-workloads/fio} time: 00:31:17 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351881/ [6351882] rbd/basic/{base/install cachepool/none clusters/{fixed-1 openstack} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} tasks/rbd_python_api_tests_old_format} time: 00:25:17 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351882/ [6351883] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-comp-lz4 validator/memcheck workloads/rbd_mirror} time: 00:48:07 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351883/ [6351884] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8.stream} workloads/rbd-mirror-workunit-policy-simple} time: 00:21:24 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351884/ [6351885] rbd/librbd/{cache/none clusters/{fixed-3 openstack} config/none min-compat-client/default msgr-failures/few objectstore/bluestore-hybrid pool/small-cache-pool supported-random-distro$/{centos_8} workloads/c_api_tests} time: 00:22:30 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351885/ [6351886] rbd/maintenance/{base/install clusters/{fixed-3 openstack} objectstore/bluestore-comp-snappy qemu/xfstests supported-random-distro$/{centos_8.stream} workloads/rebuild_object_map} time: 01:18:45 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351886/ [6351887] rbd/thrash/{base/install clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} thrashers/cache thrashosds-health workloads/rbd_fsx_rate_limit} time: 00:11:47 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351887/ [6351888] rbd/qemu/{cache/writethrough clusters/{fixed-3 openstack} features/journaling msgr-failures/few objectstore/bluestore-bitmap pool/replicated-data-pool supported-random-distro$/{centos_8.stream} workloads/qemu_bonnie} time: 00:20:58 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351888/ [6351889] rbd/cli/{base/install clusters/{fixed-1 openstack} features/layering msgr-failures/few objectstore/bluestore-comp-zstd pool/replicated-data-pool supported-random-distro$/{centos_8.stream} workloads/rbd_cli_generic} time: 00:18:52 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351889/ [6351890] rbd/encryption/{cache/writeback clusters/{fixed-3 openstack} features/defaults msgr-failures/few objectstore/bluestore-comp-zstd pool/ec-cache-pool supported-random-distro$/{rhel_8} workloads/qemu_xfstests_luks2} time: 01:16:41 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351890/ [6351891] rbd/singleton/{all/verify_pool objectstore/bluestore-comp-snappy openstack supported-random-distro$/{centos_8}} time: 00:07:26 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351891/ [6351892] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-journal-workunit} time: 00:29:36 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351892/ [6351893] rbd/cli_v1/{base/install clusters/{fixed-1 openstack} features/format-1 msgr-failures/few objectstore/bluestore-hybrid pool/small-cache-pool supported-random-distro$/{centos_8.stream} workloads/rbd_cli_import_export} time: 00:08:39 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351893/ [6351894] rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/rbd_nbd} time: 00:09:34 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351894/ [6351895] rbd/maintenance/{base/install clusters/{fixed-3 openstack} objectstore/bluestore-comp-zlib qemu/xfstests supported-random-distro$/{ubuntu_latest} workloads/dynamic_features} time: 03:07:30 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351895/ [6351896] rbd/migration/{1-base/install 2-clusters/{fixed-3 openstack} 3-objectstore/bluestore-bitmap 4-supported-random-distro$/{ubuntu_latest} 5-pool/none 6-prepare/raw-file 7-io-workloads/qemu_xfstests 8-migrate-workloads/execute 9-cleanup/cleanup} time: 02:22:26 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351896/ [6351897] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_8.stream} workloads/rbd-mirror-bootstrap-workunit} time: 00:10:00 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351897/ [6351898] rbd/singleton/{all/admin_socket objectstore/bluestore-comp-snappy openstack supported-random-distro$/{centos_8}} time: 00:07:24 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351898/ [6351899] rbd/thrash/{base/install clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/journal} time: 00:07:32 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351899/ [6351900] rbd/basic/{base/install cachepool/small clusters/{fixed-1 openstack} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8.stream} tasks/rbd_api_tests_old_format} time: 00:18:06 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351900/ [6351901] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-comp-snappy validator/memcheck workloads/c_api_tests} time: 00:36:06 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351901/ [6351902] rbd/librbd/{cache/writearound clusters/{fixed-3 openstack} config/permit-partial-discard min-compat-client/octopus msgr-failures/few objectstore/bluestore-low-osd-mem-target pool/ec-data-pool supported-random-distro$/{rhel_8} workloads/c_api_tests_with_defaults} time: 00:30:17 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351902/ [6351903] rbd/persistent-writeback-cache/{1-base/install 2-cluster/{fix-2 openstack} 3-supported-random-distro$/{rhel_8} 4-pool/cache 5-cache-mode/rwl 6-workloads/qemu_xfstests} time: 07:36:26 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351903/ [6351904] rbd/qemu/{cache/none clusters/{fixed-3 openstack} features/readbalance msgr-failures/few objectstore/bluestore-comp-lz4 pool/small-cache-pool supported-random-distro$/{centos_8.stream} workloads/qemu_fsstress} time: 00:11:25 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351904/ [6351905] rbd/cli/{base/install clusters/{fixed-1 openstack} features/defaults msgr-failures/few objectstore/bluestore-hybrid pool/small-cache-pool supported-random-distro$/{centos_8.stream} workloads/rbd_cli_groups} time: 00:08:16 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351905/ [6351906] rbd/encryption/{cache/writethrough clusters/{fixed-3 openstack} features/defaults msgr-failures/few objectstore/bluestore-hybrid pool/ec-data-pool supported-random-distro$/{centos_8} workloads/qemu_xfstests_luks1} time: 02:34:27 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351906/ [6351907] rbd/singleton/{all/formatted-output objectstore/bluestore-comp-zlib openstack supported-random-distro$/{centos_8}} time: 00:07:47 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351907/ [6351908] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8.stream} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} time: 00:48:19 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351908/ [6351909] rbd/cli_v1/{base/install clusters/{fixed-1 openstack} features/format-1 msgr-failures/few objectstore/bluestore-low-osd-mem-target pool/none supported-random-distro$/{ubuntu_latest} workloads/rbd_cli_generic} time: 00:16:22 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351909/ [6351911] rbd/thrash/{base/install clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} thrashers/cache thrashosds-health workloads/rbd_api_tests} time: 00:27:48 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351911/ [6351912] rbd/maintenance/{base/install clusters/{fixed-3 openstack} objectstore/bluestore-comp-zstd qemu/xfstests supported-random-distro$/{centos_8.stream} workloads/dynamic_features_no_cache} time: 02:02:34 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351912/ [6351913] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-ha-workunit} time: 00:13:56 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351913/ [6351914] rbd/migration/{1-base/install 2-clusters/{fixed-3 openstack} 3-objectstore/bluestore-comp-lz4 4-supported-random-distro$/{ubuntu_latest} 5-pool/replicated-data-pool 6-prepare/qcow2-file 7-io-workloads/qemu_xfstests 8-migrate-workloads/execute 9-cleanup/cleanup} time: 02:29:48 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351914/ [6351915] rbd/singleton/{all/merge_diff objectstore/bluestore-comp-zstd openstack supported-random-distro$/{ubuntu_latest}} time: 00:09:02 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351915/ [6351916] rbd/basic/{base/install cachepool/none clusters/{fixed-1 openstack} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{rhel_8} tasks/rbd_cls_tests} time: 00:15:52 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351916/ [6351917] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-comp-zlib validator/memcheck workloads/c_api_tests_with_defaults} time: 00:36:48 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351917/ [6351918] rbd/immutable-object-cache/{clusters/{fix-2 openstack} pool/ceph_and_immutable_object_cache supported-random-distro$/{ubuntu_latest} workloads/fio_on_immutable_object_cache} time: 00:22:02 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351918/ [6351919] rbd/librbd/{cache/writeback clusters/{fixed-3 openstack} config/copy-on-read min-compat-client/default msgr-failures/few objectstore/bluestore-stupid pool/none supported-random-distro$/{centos_8.stream} workloads/c_api_tests_with_journaling} time: 00:28:47 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351919/ [6351920] rbd/qemu/{cache/writearound clusters/{fixed-3 openstack} features/defaults msgr-failures/few objectstore/bluestore-comp-snappy pool/ec-cache-pool supported-random-distro$/{ubuntu_latest} workloads/qemu_xfstests} time: 02:44:52 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351920/ [6351921] rbd/thrash/{base/install clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8.stream} thrashers/default thrashosds-health workloads/rbd_api_tests_copy_on_read} time: 00:34:34 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351921/ [6351922] rbd/cli/{base/install clusters/{fixed-1 openstack} features/journaling msgr-failures/few objectstore/bluestore-low-osd-mem-target pool/ec-data-pool supported-random-distro$/{centos_8.stream} workloads/rbd_cli_import_export} time: 00:08:53 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351922/ [6351923] rbd/encryption/{cache/none clusters/{fixed-3 openstack} features/defaults msgr-failures/few objectstore/bluestore-low-osd-mem-target pool/none supported-random-distro$/{centos_8} workloads/qemu_xfstests_luks2} time: 01:55:51 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351923/ [6351924] rbd/maintenance/{base/install clusters/{fixed-3 openstack} objectstore/bluestore-hybrid qemu/xfstests supported-random-distro$/{ubuntu_latest} workloads/rebuild_object_map} time: 02:50:24 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351924/ [6351925] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8.stream} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff} time: 00:49:34 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351925/ [6351926] rbd/singleton/{all/permissions objectstore/bluestore-hybrid openstack supported-random-distro$/{ubuntu_latest}} time: 00:06:49 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351926/ [6351927] rbd/cli_v1/{base/install clusters/{fixed-1 openstack} features/format-1 msgr-failures/few objectstore/bluestore-stupid pool/small-cache-pool supported-random-distro$/{centos_8} workloads/rbd_cli_import_export} time: 00:08:35 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351927/ [6351928] rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_nbd} time: 00:11:01 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351928/ [6351929] rbd/persistent-writeback-cache/{1-base/install 2-cluster/{fix-2 openstack} 3-supported-random-distro$/{rhel_8} 4-pool/cache 5-cache-mode/ssd 6-workloads/recovery} time: 00:20:22 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351929/ [6351930] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{rhel_8} workloads/rbd-mirror-snapshot-workunit-exclusive-lock} time: 00:23:10 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351930/ [6351931] rbd/migration/{1-base/install 2-clusters/{fixed-3 openstack} 3-objectstore/bluestore-comp-snappy 4-supported-random-distro$/{ubuntu_latest} 5-pool/ec-data-pool 6-prepare/qcow2-http 7-io-workloads/qemu_xfstests 8-migrate-workloads/execute 9-cleanup/cleanup} time: 02:32:19 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351931/ [6351932] rbd/singleton/{all/qemu-iotests-no-cache objectstore/bluestore-low-osd-mem-target openstack supported-random-distro$/{centos_8.stream}} time: 00:08:43 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351932/ [6351933] rbd/thrash/{base/install clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{rhel_8} thrashers/cache thrashosds-health workloads/rbd_api_tests_journaling} time: 00:49:05 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351933/ [6351934] rbd/basic/{base/install cachepool/small clusters/{fixed-1 openstack} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8.stream} tasks/rbd_lock_and_fence} time: 00:08:33 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351934/ [6351935] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-comp-zstd validator/memcheck workloads/c_api_tests_with_journaling} time: 00:48:55 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351935/ [6351936] rbd/maintenance/{base/install clusters/{fixed-3 openstack} objectstore/bluestore-low-osd-mem-target qemu/xfstests supported-random-distro$/{centos_8.stream} workloads/dynamic_features} time: 01:20:45 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351936/ [6351937] rbd/librbd/{cache/writethrough clusters/{fixed-3 openstack} config/none min-compat-client/octopus msgr-failures/few objectstore/filestore-xfs pool/replicated-data-pool supported-random-distro$/{rhel_8} workloads/fsx} time: 00:21:13 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351937/ [6351938] rbd/qemu/{cache/writeback clusters/{fixed-3 openstack} features/journaling msgr-failures/few objectstore/bluestore-comp-zlib pool/ec-data-pool supported-random-distro$/{centos_8} workloads/qemu_bonnie} time: 00:25:34 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351938/ [6351939] rbd/cli_v1/{base/install clusters/{fixed-1 openstack} features/format-1 msgr-failures/few objectstore/filestore-xfs pool/none supported-random-distro$/{centos_8} workloads/rbd_cli_generic} time: 00:20:30 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351939/ [6351941] rbd/cli/{base/install clusters/{fixed-1 openstack} features/layering msgr-failures/few objectstore/bluestore-stupid pool/none supported-random-distro$/{centos_8.stream} workloads/rbd_cli_luks_encryption} time: 00:09:11 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351941/ [6351942] rbd/encryption/{cache/writearound clusters/{fixed-3 openstack} features/defaults msgr-failures/few objectstore/bluestore-stupid pool/replicated-data-pool supported-random-distro$/{centos_8} workloads/qemu_xfstests_luks1} time: 01:05:25 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351942/ [6351945] rbd/singleton/{all/qemu-iotests-writearound objectstore/bluestore-stupid openstack supported-random-distro$/{ubuntu_latest}} time: 00:07:09 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351945/ [6351946] rbd/thrash/{base/install clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/rbd_api_tests_no_locking} time: 00:26:46 info: http://pulpito.ceph.com/teuthology-2021-08-23_02:01:03-rbd-master-distro-basic-gibba/6351946/