2023-11-13T22:23:29.136 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-13T22:23:29.137 INFO:teuthology.results:subset: '111/120000' 2023-11-13T22:23:29.137 INFO:teuthology.results:seed: '6428' 2023-11-13T22:24:24.335 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-13T22:24:24.336 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-11-14T10:25:15.315 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds 2023-11-14T10:25:17.201 INFO:scrape:Found 24 jobs 2023-11-14T10:25:17.203 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456483/teuthology.log 2023-11-14T10:25:17.452 INFO:scrape:Found 3 distinct failure reasons 2023-11-14T10:25:17.452 INFO:scrape:Dead: None 2023-11-14T10:25:17.453 INFO:scrape:1 jobs: ['7456483'] 2023-11-14T10:25:17.453 INFO:scrape: 2023-11-14T10:25:17.454 INFO:scrape:Failure: Command failed on smithi044 with status 1: 'sudo fuser -v /var/lib/dpkg/lock-frontend' 2023-11-14T10:25:17.454 INFO:scrape:1 jobs: ['7456479'] 2023-11-14T10:25:17.454 INFO:scrape:suites: ['objectstore/bluestore-bitmap', 'openstack}', 's3tests-branch', 'smoke/basic/{clusters/{fixed-3-cephfs', 'supported-random-distro$/{ubuntu_20.04}', 'tasks/{0-install', 'test/kclient_workunit_suites_pjd}}'] 2023-11-14T10:25:17.455 INFO:scrape: 2023-11-14T10:25:17.455 INFO:scrape:Dead: b'2023-11-13T22:55:09.375 INFO:teuthology.orchestra.run.smithi093.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 13 22:55 /home/ubuntu/cephtest' 2023-11-14T10:25:17.456 INFO:scrape:1 jobs: ['7456476'] 2023-11-14T10:25:17.456 INFO:scrape:suites: ['objectstore/bluestore-bitmap', 'openstack}', 's3tests-branch', 'smoke/basic/{clusters/{fixed-3-cephfs', 'supported-random-distro$/{ubuntu_20.04}', 'tasks/{0-install', 'test/kclient_workunit_direct_io}}'] 2023-11-14T10:25:17.456 INFO:scrape: 2023-11-14T10:25:17.457 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi ================================================================= info: https://pulpito.ceph.com/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/ failed: 1 dead: 1 running: 1 waiting: 0 queued: 0 passed: 21 Fail ================================================================= [7456479] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{ubuntu_20.04} tasks/{0-install test/kclient_workunit_suites_pjd}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456479/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456479/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c093d1e185274d1f8039fe5dc706cb82 Command failed on smithi044 with status 1: 'sudo fuser -v /var/lib/dpkg/lock-frontend' Dead ================================================================= [7456483] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{rhel_8} tasks/{0-install test/rados_bench}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456483/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456483/ Error reimaging machines: reached maximum tries (241) after waiting for 3600 seconds Running ================================================================= [7456476] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{ubuntu_20.04} tasks/{0-install test/kclient_workunit_direct_io}} info: https://pulpito.ceph.com/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456476/ Pass ================================================================= [7456472] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{ubuntu_latest} tasks/{0-install test/cfuse_workunit_suites_blogbench}} time: 00:23:45 info: https://pulpito.ceph.com/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456472/ [7456473] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{centos_8} tasks/{0-install test/cfuse_workunit_suites_fsstress}} time: 00:16:59 info: https://pulpito.ceph.com/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456473/ [7456474] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{centos_8} tasks/{0-install test/cfuse_workunit_suites_iozone}} time: 00:20:30 info: https://pulpito.ceph.com/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456474/ [7456475] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{centos_latest} tasks/{0-install test/cfuse_workunit_suites_pjd}} time: 00:15:59 info: https://pulpito.ceph.com/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456475/ [7456477] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{rhel_8} tasks/{0-install test/kclient_workunit_suites_dbench}} time: 00:53:22 info: https://pulpito.ceph.com/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456477/ [7456478] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{ubuntu_latest} tasks/{0-install test/kclient_workunit_suites_fsstress}} time: 00:17:15 info: https://pulpito.ceph.com/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456478/ [7456480] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{centos_latest} tasks/{0-install test/libcephfs_interface_tests}} time: 00:25:04 info: https://pulpito.ceph.com/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456480/ [7456481] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{ubuntu_latest} tasks/{0-install test/mon_thrash}} time: 00:37:18 info: https://pulpito.ceph.com/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456481/ [7456482] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{ubuntu_20.04} tasks/{0-install test/rados_api_tests}} time: 00:31:37 info: https://pulpito.ceph.com/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456482/ [7456484] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{rhel_8} tasks/{0-install test/rados_cache_snaps}} time: 00:36:09 info: https://pulpito.ceph.com/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456484/ [7456485] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rados_cls_all}} time: 00:28:11 info: https://pulpito.ceph.com/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456485/ [7456486] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{ubuntu_20.04} tasks/{0-install test/rados_ec_snaps}} time: 00:28:26 info: https://pulpito.ceph.com/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456486/ [7456487] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rados_python}} time: 00:23:46 info: https://pulpito.ceph.com/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456487/ [7456488] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{rhel_8} tasks/{0-install test/rados_workunit_loadgen_mix}} time: 00:36:33 info: https://pulpito.ceph.com/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456488/ [7456489] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{centos_latest} tasks/{0-install test/rbd_api_tests}} time: 00:48:21 info: https://pulpito.ceph.com/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456489/ [7456490] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rbd_cli_import_export}} time: 00:17:27 info: https://pulpito.ceph.com/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456490/ [7456491] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{ubuntu_20.04} tasks/{0-install test/rbd_fsx}} time: 00:15:35 info: https://pulpito.ceph.com/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456491/ [7456492] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{rhel_8} tasks/{0-install test/rbd_python_api_tests}} time: 00:32:11 info: https://pulpito.ceph.com/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456492/ [7456493] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{ubuntu_20.04} tasks/{0-install test/rbd_workunit_suites_iozone}} time: 00:21:00 info: https://pulpito.ceph.com/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456493/ [7456494] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{centos_latest} tasks/{0-install test/rgw_ec_s3tests}} time: 00:25:31 info: https://pulpito.ceph.com/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456494/ [7456495] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap s3tests-branch supported-random-distro$/{rhel_8} tasks/{0-install test/rgw_s3tests}} time: 00:32:02 info: https://pulpito.ceph.com/yuriw-2023-11-13_22:23:04-smoke-reef-release-distro-default-smithi/7456495/