2022-08-15T13:13:06.225 INFO:root:teuthology version: 1.1.1.dev409+g9e7483cc
2022-08-15T13:13:06.227 INFO:teuthology.results:subset: '64/99'
2022-08-15T13:13:06.227 INFO:teuthology.results:seed: '7599'
2022-08-15T13:34:36.870 INFO:root:teuthology version: 1.1.1.dev409+g9e7483cc
2022-08-15T13:34:36.873 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2022-08-16T01:35:31.786 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds
2022-08-16T01:35:33.336 INFO:scrape:Found 17 jobs
2022-08-16T01:35:35.081 INFO:scrape:Found 2 distinct failure reasons
2022-08-16T01:35:35.083 INFO:scrape:Failure: "2022-08-15T13:35:26.586733+0000 mon.a (mon.0) 200 : cluster [WRN] Health check failed: Degraded data redundancy: 1/1964 objects degraded (0.051%), 1 pg degraded (PG_DEGRADED)" in cluster log
2022-08-16T01:35:35.083 INFO:scrape:3 jobs: ['6973360', '6973371', '6973374']
2022-08-16T01:35:35.084 INFO:scrape:suites intersection: ['clients/mirror', 'cluster/{2-node', 'msgr-failures/few', 'openstack}', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install']
2022-08-16T01:35:35.085 INFO:scrape:suites union: ['clients/mirror', 'cluster/{2-node', 'msgr-failures/few', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'policy/none', 'policy/simple', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rbd-mirror-fsx-workunit}', 'workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}']
2022-08-16T01:35:35.085 INFO:scrape:
2022-08-16T01:35:35.085 INFO:scrape:Dead: b'2022-08-16T01:31:56.321 DEBUG:teuthology.exit:Finished running handlers'
2022-08-16T01:35:35.086 INFO:scrape:1 jobs: ['6973368']
2022-08-16T01:35:35.086 INFO:scrape:suites: ['clusters/{fixed-3', 'objectstore/bluestore-hybrid', 'openstack}', 'qemu/xfstests', 'rbd/maintenance/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'workloads/dynamic_features_no_cache}']
2022-08-16T01:35:35.087 INFO:scrape:
2022-08-16T01:35:35.088 INFO:teuthology.results:Sending results to idryomov@gmail.com: Test Run: dis-2022-08-15_13:12:48-rbd-wip-dis-testing-1-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/dis-2022-08-15_13:12:48-rbd-wip-dis-testing-1-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/dis-2022-08-15_13:12:48-rbd-wip-dis-testing-1-distro-default-smithi/
failed:  3
dead:    0
running: 1
waiting: 0
queued:  0
passed:  13


Fail
=================================================================
[6973360]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid policy/none rbd-mirror/four-per-cluster supported-random-distro$/{rhel_8} workloads/rbd-mirror-fsx-workunit}
-----------------------------------------------------------------
time:   00:39:54
info:   http://pulpito.ceph.com/dis-2022-08-15_13:12:48-rbd-wip-dis-testing-1-distro-default-smithi/6973360/
log:    http://qa-proxy.ceph.com/teuthology/dis-2022-08-15_13:12:48-rbd-wip-dis-testing-1-distro-default-smithi/6973360/

    "2022-08-15T13:35:26.586733+0000 mon.a (mon.0) 200 : cluster [WRN] Health
    check failed: Degraded data redundancy: 1/1964 objects degraded (0.051%), 1
    pg degraded (PG_DEGRADED)" in cluster log

[6973371]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
-----------------------------------------------------------------
time:   00:46:48
info:   http://pulpito.ceph.com/dis-2022-08-15_13:12:48-rbd-wip-dis-testing-1-distro-default-smithi/6973371/
log:    http://qa-proxy.ceph.com/teuthology/dis-2022-08-15_13:12:48-rbd-wip-dis-testing-1-distro-default-smithi/6973371/

    "2022-08-15T13:39:42.882100+0000 mon.a (mon.0) 104 : cluster [WRN] Health
    check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1
    pg degraded (PG_DEGRADED)" in cluster log

[6973374]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-fsx-workunit}
-----------------------------------------------------------------
time:   00:36:29
info:   http://pulpito.ceph.com/dis-2022-08-15_13:12:48-rbd-wip-dis-testing-1-distro-default-smithi/6973374/
log:    http://qa-proxy.ceph.com/teuthology/dis-2022-08-15_13:12:48-rbd-wip-dis-testing-1-distro-default-smithi/6973374/

    "2022-08-15T13:49:17.374436+0000 mon.a (mon.0) 226 : cluster [WRN] Health
    check failed: Degraded data redundancy: 1/1974 objects degraded (0.051%), 1
    pg degraded (PG_DEGRADED)" in cluster log



Running
=================================================================
[6973368] rbd/maintenance/{base/install clusters/{fixed-3 openstack} objectstore/bluestore-hybrid qemu/xfstests supported-random-distro$/{ubuntu_latest} workloads/dynamic_features_no_cache}
info:   http://pulpito.ceph.com/dis-2022-08-15_13:12:48-rbd-wip-dis-testing-1-distro-default-smithi/6973368/



Pass
=================================================================
[6973358] rbd/migration/{1-base/install 2-clusters/{fixed-3 openstack} 3-objectstore/bluestore-hybrid 4-supported-random-distro$/{rhel_8} 5-pool/none 6-prepare/raw-file 7-io-workloads/qemu_xfstests 8-migrate-workloads/execute 9-cleanup/cleanup}
time:   01:23:10
info:   http://pulpito.ceph.com/dis-2022-08-15_13:12:48-rbd-wip-dis-testing-1-distro-default-smithi/6973358/

[6973359] rbd/librbd/{cache/writeback clusters/{fixed-3 openstack} config/none min-compat-client/default msgr-failures/few objectstore/bluestore-comp-lz4 pool/small-cache-pool supported-random-distro$/{ubuntu_latest} workloads/c_api_tests_with_defaults}
time:   00:26:20
info:   http://pulpito.ceph.com/dis-2022-08-15_13:12:48-rbd-wip-dis-testing-1-distro-default-smithi/6973359/

[6973361] rbd/librbd/{cache/writearound clusters/{fixed-3 openstack} config/copy-on-read min-compat-client/octopus msgr-failures/few objectstore/bluestore-comp-zlib pool/replicated-data-pool supported-random-distro$/{rhel_8} workloads/python_api_tests}
time:   00:24:08
info:   http://pulpito.ceph.com/dis-2022-08-15_13:12:48-rbd-wip-dis-testing-1-distro-default-smithi/6973361/

[6973362] rbd/cli/{base/install clusters/{fixed-1 openstack} features/layering msgr-failures/few objectstore/bluestore-comp-zstd pool/replicated-data-pool supported-random-distro$/{rhel_8} workloads/rbd_cli_import_export}
time:   00:13:30
info:   http://pulpito.ceph.com/dis-2022-08-15_13:12:48-rbd-wip-dis-testing-1-distro-default-smithi/6973362/

[6973363] rbd/librbd/{cache/none clusters/{fixed-3 openstack} config/copy-on-read min-compat-client/default msgr-failures/few objectstore/bluestore-low-osd-mem-target pool/none supported-random-distro$/{ubuntu_latest} workloads/rbd_fio}
time:   00:25:40
info:   http://pulpito.ceph.com/dis-2022-08-15_13:12:48-rbd-wip-dis-testing-1-distro-default-smithi/6973363/

[6973364] rbd/librbd/{cache/writearound clusters/{fixed-3 openstack} config/copy-on-read min-compat-client/octopus msgr-failures/few objectstore/bluestore-low-osd-mem-target pool/replicated-data-pool supported-random-distro$/{rhel_8} workloads/c_api_tests}
time:   00:29:07
info:   http://pulpito.ceph.com/dis-2022-08-15_13:12:48-rbd-wip-dis-testing-1-distro-default-smithi/6973364/

[6973365] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-fsx-workunit}
time:   00:38:50
info:   http://pulpito.ceph.com/dis-2022-08-15_13:12:48-rbd-wip-dis-testing-1-distro-default-smithi/6973365/

[6973366] rbd/pwl-cache/tmpfs/{1-base/install 2-cluster/{fix-2 openstack} 3-supported-random-distro$/{rhel_8} 4-cache-path 5-cache-mode/ssd 6-cache-size/5G 7-workloads/qemu_xfstests}
time:   01:02:44
info:   http://pulpito.ceph.com/dis-2022-08-15_13:12:48-rbd-wip-dis-testing-1-distro-default-smithi/6973366/

[6973367] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-journal-bootstrap-workunit}
time:   00:08:47
info:   http://pulpito.ceph.com/dis-2022-08-15_13:12:48-rbd-wip-dis-testing-1-distro-default-smithi/6973367/

[6973369] rbd/librbd/{cache/writethrough clusters/{fixed-3 openstack} config/none min-compat-client/octopus msgr-failures/few objectstore/bluestore-stupid pool/ec-data-pool supported-random-distro$/{ubuntu_latest} workloads/c_api_tests_with_journaling}
time:   00:33:12
info:   http://pulpito.ceph.com/dis-2022-08-15_13:12:48-rbd-wip-dis-testing-1-distro-default-smithi/6973369/

[6973370] rbd/librbd/{cache/none clusters/{fixed-3 openstack} config/copy-on-read min-compat-client/default msgr-failures/few objectstore/bluestore-comp-zlib pool/none supported-random-distro$/{centos_8} workloads/rbd_fio}
time:   00:26:51
info:   http://pulpito.ceph.com/dis-2022-08-15_13:12:48-rbd-wip-dis-testing-1-distro-default-smithi/6973370/

[6973372] rbd/qemu/{cache/writeback clusters/{fixed-3 openstack} features/journaling msgr-failures/few objectstore/bluestore-hybrid pool/replicated-data-pool supported-random-distro$/{rhel_8} workloads/qemu_bonnie}
time:   00:25:29
info:   http://pulpito.ceph.com/dis-2022-08-15_13:12:48-rbd-wip-dis-testing-1-distro-default-smithi/6973372/

[6973373] rbd/qemu/{cache/none clusters/{fixed-3 openstack} features/defaults msgr-failures/few objectstore/bluestore-stupid pool/ec-cache-pool supported-random-distro$/{ubuntu_latest} workloads/qemu_xfstests}
time:   03:38:26
info:   http://pulpito.ceph.com/dis-2022-08-15_13:12:48-rbd-wip-dis-testing-1-distro-default-smithi/6973373/