2022-05-15T05:00:51.686 INFO:root:teuthology version: 1.1.1.dev333+gb018fa01 2022-05-15T05:00:51.689 INFO:teuthology.results:seed: '3100' 2022-05-15T05:16:29.900 INFO:root:teuthology version: 1.1.1.dev333+gb018fa01 2022-05-15T05:16:29.904 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-05-15T06:18:34.786 INFO:teuthology.results:Tests finished! gathering results... 2022-05-15T06:18:36.164 INFO:scrape:Found 24 jobs 2022-05-15T06:18:37.014 INFO:scrape:Found 3 distinct failure reasons 2022-05-15T06:18:37.015 INFO:scrape:Failure: Command failed on smithi079 with status 1: 'sudo yum install -y kernel' 2022-05-15T06:18:37.016 INFO:scrape:5 jobs: ['6834308', '6834293', '6834292', '6834296', '6834301'] 2022-05-15T06:18:37.016 INFO:scrape:suites intersection: ['objectstore/bluestore-bitmap', 'openstack}', 'smoke/basic/{clusters/{fixed-3-cephfs', 'supported-random-distro$/{centos_8}', 'tasks/{0-install'] 2022-05-15T06:18:37.017 INFO:scrape:suites union: ['objectstore/bluestore-bitmap', 'openstack}', 'smoke/basic/{clusters/{fixed-3-cephfs', 'supported-random-distro$/{centos_8}', 'tasks/{0-install', 'test/cfuse_workunit_suites_iozone}}', 'test/cfuse_workunit_suites_pjd}}', 'test/kclient_workunit_suites_fsstress}}', 'test/rados_bench}}', 'test/rbd_cli_import_export}}'] 2022-05-15T06:18:37.017 INFO:scrape: 2022-05-15T06:18:37.018 INFO:scrape:Failure: "2022-05-15T05:29:17.562420+0000 mon.a (mon.0) 258 : cluster [WRN] Health check failed: Degraded data redundancy: 1/50 objects degraded (2.000%), 1 pg degraded (PG_DEGRADED)" in cluster log 2022-05-15T06:18:37.018 INFO:scrape:1 jobs: ['6834303'] 2022-05-15T06:18:37.018 INFO:scrape:suites: ['objectstore/bluestore-bitmap', 'openstack}', 'smoke/basic/{clusters/{fixed-3-cephfs', 'supported-random-distro$/{ubuntu_latest}', 'tasks/{0-install', 'test/rados_cls_all}}'] 2022-05-15T06:18:37.019 INFO:scrape: 2022-05-15T06:18:37.019 INFO:scrape:Failure: reached maximum tries (90) after waiting for 540 seconds 2022-05-15T06:18:37.020 INFO:scrape:1 jobs: ['6834299'] 2022-05-15T06:18:37.020 INFO:scrape:suites: ['objectstore/bluestore-bitmap', 'openstack}', 'smoke/basic/{clusters/{fixed-3-cephfs', 'supported-random-distro$/{rhel_8}', 'tasks/{0-install', 'test/mon_thrash}}'] 2022-05-15T06:18:37.021 INFO:scrape: 2022-05-15T06:18:37.022 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/ failed: 7 dead: 0 running: 0 waiting: 0 queued: 0 passed: 17 Fail ================================================================= [6834292] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/cfuse_workunit_suites_iozone}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834292/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834292/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cdaf3963eb804d83abea88f84ef653c9 Command failed on smithi204 with status 1: 'sudo yum install -y kernel' [6834293] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/cfuse_workunit_suites_pjd}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834293/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834293/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d52527a0263d44199bdbea14838619c1 Command failed on smithi115 with status 1: 'sudo yum install -y kernel' [6834296] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/kclient_workunit_suites_fsstress}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834296/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834296/ sentry: https://sentry.ceph.com/organizations/ceph/?query=eb1f09c69c5a45f5855f4ccc6d723e23 Command failed on smithi125 with status 1: 'sudo yum install -y kernel' [6834299] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/mon_thrash}} ----------------------------------------------------------------- time: 00:23:31 info: http://pulpito.ceph.com/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834299/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834299/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1cadb2b978d3462d9560a49463a0f5e6 reached maximum tries (90) after waiting for 540 seconds [6834301] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/rados_bench}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834301/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834301/ sentry: https://sentry.ceph.com/organizations/ceph/?query=524434eecbf1475c819f0b69e4cbeace Command failed on smithi174 with status 1: 'sudo yum install -y kernel' [6834303] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rados_cls_all}} ----------------------------------------------------------------- time: 00:22:23 info: http://pulpito.ceph.com/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834303/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834303/ "2022-05-15T05:29:17.562420+0000 mon.a (mon.0) 258 : cluster [WRN] Health check failed: Degraded data redundancy: 1/50 objects degraded (2.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6834308] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/rbd_cli_import_export}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834308/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834308/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f5dedc5a626e444785113683c89ba715 Command failed on smithi079 with status 1: 'sudo yum install -y kernel' Pass ================================================================= [6834290] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/cfuse_workunit_suites_blogbench}} time: 00:22:48 info: http://pulpito.ceph.com/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834290/ [6834291] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/cfuse_workunit_suites_fsstress}} time: 00:14:30 info: http://pulpito.ceph.com/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834291/ [6834294] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/kclient_workunit_direct_io}} time: 00:17:26 info: http://pulpito.ceph.com/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834294/ [6834295] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/kclient_workunit_suites_dbench}} time: 01:04:47 info: http://pulpito.ceph.com/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834295/ [6834297] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/kclient_workunit_suites_pjd}} time: 00:13:44 info: http://pulpito.ceph.com/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834297/ [6834298] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/libcephfs_interface_tests}} time: 00:15:26 info: http://pulpito.ceph.com/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834298/ [6834300] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rados_api_tests}} time: 00:30:02 info: http://pulpito.ceph.com/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834300/ [6834302] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rados_cache_snaps}} time: 00:27:07 info: http://pulpito.ceph.com/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834302/ [6834304] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rados_ec_snaps}} time: 00:27:52 info: http://pulpito.ceph.com/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834304/ [6834305] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rados_python}} time: 00:18:27 info: http://pulpito.ceph.com/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834305/ [6834306] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/rados_workunit_loadgen_mix}} time: 00:27:33 info: http://pulpito.ceph.com/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834306/ [6834307] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rbd_api_tests}} time: 00:31:26 info: http://pulpito.ceph.com/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834307/ [6834309] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/rbd_fsx}} time: 00:20:28 info: http://pulpito.ceph.com/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834309/ [6834310] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rbd_python_api_tests}} time: 00:23:46 info: http://pulpito.ceph.com/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834310/ [6834311] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/rbd_workunit_suites_iozone}} time: 00:21:52 info: http://pulpito.ceph.com/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834311/ [6834312] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/rgw_ec_s3tests}} time: 00:26:24 info: http://pulpito.ceph.com/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834312/ [6834313] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/rgw_s3tests}} time: 00:26:03 info: http://pulpito.ceph.com/teuthology-2022-05-15_05:00:04-smoke-master-distro-default-smithi/6834313/