2023-11-04T03:46:38.053 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-04T03:46:38.054 INFO:teuthology.results:subset: '111/120000' 2023-11-04T03:46:38.054 INFO:teuthology.results:seed: '1832' 2023-11-04T04:02:33.634 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-04T04:02:33.637 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-11-04T04:45:37.081 INFO:teuthology.results:Tests finished! gathering results... 2023-11-04T04:45:37.390 INFO:scrape:Found 24 jobs 2023-11-04T04:45:37.391 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446746/teuthology.log 2023-11-04T04:45:37.721 INFO:scrape:Found 2 distinct failure reasons 2023-11-04T04:45:37.722 INFO:scrape:Dead: None 2023-11-04T04:45:37.723 INFO:scrape:18 jobs: ['7446746', '7446782', '7446761', '7446729', '7446735', '7446759', '7446743', '7446770', '7446740', '7446738', '7446785', '7446764', '7446756', '7446773', '7446788', '7446732', '7446776', '7446779'] 2023-11-04T04:45:37.723 INFO:scrape: 2023-11-04T04:45:37.724 INFO:scrape:Failure: "2023-11-04T04:09:52.647282+0000 mon.a (mon.0) 127 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log 2023-11-04T04:45:37.724 INFO:scrape:1 jobs: ['7446753'] 2023-11-04T04:45:37.725 INFO:scrape:suites: ['objectstore/bluestore-bitmap', 'openstack}', 's3tests-branch', 'smoke/basic/{clusters/{fixed-3-cephfs', 'supported-random-distro$/{centos_latest}', 'tasks/{0-install', 'test/libcephfs_interface_tests}}'] 2023-11-04T04:45:37.725 INFO:scrape: 2023-11-04T04:45:37.726 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/ failed: 1 dead: 18 running: 0 waiting: 0 queued: 0 passed: 5 Fail ================================================================= [7446753] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{centos_latest} tasks/{0-install test/libcephfs_interface_tests}} ----------------------------------------------------------------- time: 00:23:08 info: http://pulpito.ceph.com/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446753/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446753/ "2023-11-04T04:09:52.647282+0000 mon.a (mon.0) 127 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log Dead ================================================================= [7446729] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{ubuntu_20.04} tasks/{0-install test/cfuse_workunit_suites_blogbench}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446729/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446729/ Error reimaging machines: Failed to power on smithi124 [7446732] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{ubuntu_latest} tasks/{0-install test/cfuse_workunit_suites_fsstress}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446732/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446732/ Error reimaging machines: Failed to power on smithi103 [7446735] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{centos_latest} tasks/{0-install test/cfuse_workunit_suites_iozone}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446735/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446735/ Error reimaging machines: Failed to power on smithi112 [7446738] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{rhel_8} tasks/{0-install test/cfuse_workunit_suites_pjd}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446738/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446738/ Error reimaging machines: Failed to power on smithi111 [7446740] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{rhel_8} tasks/{0-install test/kclient_workunit_direct_io}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446740/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446740/ Error reimaging machines: Failed to power on smithi123 [7446743] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{centos_latest} tasks/{0-install test/kclient_workunit_suites_dbench}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446743/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446743/ Error reimaging machines: Failed to power on smithi114 [7446746] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{centos_latest} tasks/{0-install test/kclient_workunit_suites_fsstress}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446746/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446746/ Error reimaging machines: Failed to power on smithi106 [7446756] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{ubuntu_latest} tasks/{0-install test/mon_thrash}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446756/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446756/ Error reimaging machines: Failed to power on smithi121 [7446759] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{centos_8} tasks/{0-install test/rados_api_tests}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446759/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446759/ Error reimaging machines: Failed to power on smithi124 [7446761] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rados_bench}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446761/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446761/ Error reimaging machines: Failed to power on smithi112 [7446764] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{rhel_8} tasks/{0-install test/rados_cache_snaps}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446764/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446764/ Error reimaging machines: Failed to power on smithi111 [7446770] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{centos_latest} tasks/{0-install test/rados_ec_snaps}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446770/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446770/ Error reimaging machines: Failed to power on smithi099 [7446773] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{rhel_8} tasks/{0-install test/rados_python}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446773/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446773/ Error reimaging machines: Failed to power on smithi114 [7446776] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{ubuntu_20.04} tasks/{0-install test/rados_workunit_loadgen_mix}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446776/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446776/ Error reimaging machines: Failed to power on smithi118 [7446779] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{centos_8} tasks/{0-install test/rbd_api_tests}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446779/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446779/ Error reimaging machines: Failed to power on smithi106 [7446782] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{rhel_8} tasks/{0-install test/rbd_cli_import_export}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446782/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446782/ Error reimaging machines: Failed to power on smithi121 [7446785] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rbd_fsx}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446785/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446785/ Error reimaging machines: Failed to power on smithi124 [7446788] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rbd_python_api_tests}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446788/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446788/ Error reimaging machines: Failed to power on smithi103 Pass ================================================================= [7446749] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{ubuntu_latest} tasks/{0-install test/kclient_workunit_suites_pjd}} time: 00:16:58 info: http://pulpito.ceph.com/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446749/ [7446768] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{rhel_8} tasks/{0-install test/rados_cls_all}} time: 00:28:00 info: http://pulpito.ceph.com/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446768/ [7446791] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{rhel_8} tasks/{0-install test/rbd_workunit_suites_iozone}} time: 00:25:59 info: http://pulpito.ceph.com/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446791/ [7446794] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{centos_8} tasks/{0-install test/rgw_ec_s3tests}} time: 00:25:17 info: http://pulpito.ceph.com/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446794/ [7446797] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{rhel_8} tasks/{0-install test/rgw_s3tests}} time: 00:31:03 info: http://pulpito.ceph.com/yuriw-2023-11-04_03:36:46-smoke-reef-release-distro-default-smithi/7446797/