2022-07-10T05:01:11.767 INFO:root:teuthology version: 1.1.1.dev381+g10062088 2022-07-10T05:01:11.783 INFO:teuthology.results:seed: '7277' 2022-07-10T05:21:50.579 INFO:root:teuthology version: 1.1.1.dev381+g10062088 2022-07-10T05:21:50.614 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-07-10T06:27:57.719 INFO:teuthology.results:Tests finished! gathering results... 2022-07-10T06:28:00.914 INFO:scrape:Found 24 jobs 2022-07-10T06:28:04.060 INFO:scrape:Found 4 distinct failure reasons 2022-07-10T06:28:04.061 INFO:scrape:Failure: {'smithi089.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi112.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi179.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} 2022-07-10T06:28:04.061 INFO:scrape:5 jobs: ['6922050', '6922071', '6922066', '6922070', '6922055'] 2022-07-10T06:28:04.062 INFO:scrape:suites intersection: ['objectstore/bluestore-bitmap', 'openstack}', 'smoke/basic/{clusters/{fixed-3-cephfs', 'supported-random-distro$/{rhel_8}', 'tasks/{0-install'] 2022-07-10T06:28:04.071 INFO:scrape:suites union: ['objectstore/bluestore-bitmap', 'openstack}', 'smoke/basic/{clusters/{fixed-3-cephfs', 'supported-random-distro$/{rhel_8}', 'tasks/{0-install', 'test/cfuse_workunit_suites_iozone}}', 'test/kclient_workunit_suites_pjd}}', 'test/rbd_cli_import_export}}', 'test/rgw_ec_s3tests}}', 'test/rgw_s3tests}}'] 2022-07-10T06:28:04.072 INFO:scrape: 2022-07-10T06:28:04.083 INFO:scrape:Failure: Command failed (workunit test libcephfs/test.sh) on smithi132 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=33f576a9c09ff3a3ee05d050ca8d49556340221f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/libcephfs/test.sh' 2022-07-10T06:28:04.083 INFO:scrape:1 jobs: ['6922056'] 2022-07-10T06:28:04.084 INFO:scrape:suites: ['objectstore/bluestore-bitmap', 'openstack}', 'smoke/basic/{clusters/{fixed-3-cephfs', 'supported-random-distro$/{centos_8}', 'tasks/{0-install', 'test/libcephfs_interface_tests}}'] 2022-07-10T06:28:04.085 INFO:scrape: 2022-07-10T06:28:04.085 INFO:scrape:Failure: {'smithi057.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'dest': '/tmp/katello-ca-consumer-latest.noarch.rpm', 'elapsed': 10, 'gid': 0, 'group': 'root', 'invocation': {'module_args': {'attributes': None, 'backup': False, 'checksum': '', 'client_cert': None, 'client_key': None, 'dest': '/tmp/katello-ca-consumer-latest.noarch.rpm', 'force': False, 'force_basic_auth': False, 'group': None, 'headers': None, 'http_agent': 'ansible-httpget', 'mode': None, 'owner': None, 'selevel': None, 'serole': None, 'setype': None, 'seuser': None, 'sha256sum': '', 'timeout': 10, 'tmp_dest': None, 'unsafe_writes': False, 'url': 'http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-latest.noarch.rpm', 'url_password': None, 'url_username': None, 'use_proxy': True, 'validate_certs': True}}, 'mode': '0644', 'msg': 'Connection failure: timed out', 'owner': 'root', 'secontext': 'unconfined_u:object_r:user_home_t:s0', 'size': 7419, 'state': 'file', 'uid': 0, 'url': 'http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer-latest.noarch.rpm'}, 'smithi077.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi086.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} 2022-07-10T06:28:04.085 INFO:scrape:1 jobs: ['6922058'] 2022-07-10T06:28:04.086 INFO:scrape:suites: ['objectstore/bluestore-bitmap', 'openstack}', 'smoke/basic/{clusters/{fixed-3-cephfs', 'supported-random-distro$/{rhel_8}', 'tasks/{0-install', 'test/rados_api_tests}}'] 2022-07-10T06:28:04.086 INFO:scrape: 2022-07-10T06:28:04.087 INFO:scrape:Failure: reached maximum tries (90) after waiting for 540 seconds 2022-07-10T06:28:04.088 INFO:scrape:1 jobs: ['6922059'] 2022-07-10T06:28:04.088 INFO:scrape:suites: ['objectstore/bluestore-bitmap', 'openstack}', 'smoke/basic/{clusters/{fixed-3-cephfs', 'supported-random-distro$/{ubuntu_latest}', 'tasks/{0-install', 'test/rados_bench}}'] 2022-07-10T06:28:04.089 INFO:scrape: 2022-07-10T06:28:04.090 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/ failed: 2 dead: 6 running: 0 waiting: 0 queued: 0 passed: 16 Fail ================================================================= [6922056] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/libcephfs_interface_tests}} ----------------------------------------------------------------- time: 00:17:26 info: http://pulpito.ceph.com/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922056/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922056/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7a2091e1650e47d18ba9ef77c466cc57 Command failed (workunit test libcephfs/test.sh) on smithi132 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=33f576a9c09ff3a3ee05d050ca8d49556340221f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/libcephfs/test.sh' [6922059] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rados_bench}} ----------------------------------------------------------------- time: 00:19:40 info: http://pulpito.ceph.com/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922059/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922059/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9b8f3bf54f2b404881d5b0727d02eb98 reached maximum tries (90) after waiting for 540 seconds Dead ================================================================= [6922050] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/cfuse_workunit_suites_iozone}} ----------------------------------------------------------------- time: 00:20:27 info: http://pulpito.ceph.com/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922050/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922050/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9cef896cb7a04c1081d98f609e46179a {'smithi089.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi112.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi179.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} [6922055] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/kclient_workunit_suites_pjd}} ----------------------------------------------------------------- time: 00:19:54 info: http://pulpito.ceph.com/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922055/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922055/ sentry: https://sentry.ceph.com/organizations/ceph/?query=718ca850de1444cb8e5ea89c34a15f17 {'smithi156.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi080.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi003.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} [6922058] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/rados_api_tests}} ----------------------------------------------------------------- time: 00:14:50 info: http://pulpito.ceph.com/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922058/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922058/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ae396957673f41bf9e9722d803ba357f {'smithi057.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'dest': '/tmp/katello-ca-consumer-latest.noarch.rpm', 'elapsed': 10, 'gid': 0, 'group': 'root', 'invocation': {'module_args': {'attributes': None, 'backup': False, 'checksum': '', 'client_cert': None, 'client_key': None, 'dest': '/tmp/katello-ca-consumer-latest.noarch.rpm', 'force': False, 'force_basic_auth': False, 'group': None, 'headers': None, 'http_agent': 'ansible-httpget', 'mode': None, 'owner': None, 'selevel': None, 'serole': None, 'setype': None, 'seuser': None, 'sha256sum': '', 'timeout': 10, 'tmp_dest': None, 'unsafe_writes': False, 'url': 'http://satellite.front.sepia.ceph.com/pub/katello-ca-consumer- latest.noarch.rpm', 'url_password': None, 'url_username': None, 'use_proxy': True, 'validate_certs': True}}, 'mode': '0644', 'msg': 'Connection failure: timed out', 'owner': 'root', 'secontext': 'unconfined_u:object_r:user_home_t:s0', 'size': 7419, 'state': 'file', 'uid': 0, 'url': 'http://satellite.front.sepia.ceph.com/pub/katello-ca- consumer-latest.noarch.rpm'}, 'smithi077.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi086.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} [6922066] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/rbd_cli_import_export}} ----------------------------------------------------------------- time: 00:06:04 info: http://pulpito.ceph.com/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922066/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922066/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ccef5c18879a4a8f9203f9e4b99a5394 {'smithi174.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi115.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi066.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} [6922070] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/rgw_ec_s3tests}} ----------------------------------------------------------------- time: 00:05:57 info: http://pulpito.ceph.com/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922070/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922070/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e30db96494344a609ca48228bdc10eb1 {'smithi137.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi142.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi125.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} [6922071] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/rgw_s3tests}} ----------------------------------------------------------------- time: 00:06:06 info: http://pulpito.ceph.com/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922071/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922071/ sentry: https://sentry.ceph.com/organizations/ceph/?query=49b17ccdda5440e3906ca151c9d52244 {'smithi172.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi043.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi001.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} Pass ================================================================= [6922048] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/cfuse_workunit_suites_blogbench}} time: 00:22:00 info: http://pulpito.ceph.com/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922048/ [6922049] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/cfuse_workunit_suites_fsstress}} time: 00:16:26 info: http://pulpito.ceph.com/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922049/ [6922051] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/cfuse_workunit_suites_pjd}} time: 00:17:28 info: http://pulpito.ceph.com/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922051/ [6922052] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/kclient_workunit_direct_io}} time: 00:14:48 info: http://pulpito.ceph.com/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922052/ [6922053] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/kclient_workunit_suites_dbench}} time: 01:18:26 info: http://pulpito.ceph.com/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922053/ [6922054] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/kclient_workunit_suites_fsstress}} time: 00:16:35 info: http://pulpito.ceph.com/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922054/ [6922057] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/mon_thrash}} time: 00:35:21 info: http://pulpito.ceph.com/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922057/ [6922060] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/rados_cache_snaps}} time: 00:29:38 info: http://pulpito.ceph.com/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922060/ [6922061] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rados_cls_all}} time: 00:24:33 info: http://pulpito.ceph.com/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922061/ [6922062] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rados_ec_snaps}} time: 00:28:36 info: http://pulpito.ceph.com/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922062/ [6922063] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rados_python}} time: 00:18:52 info: http://pulpito.ceph.com/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922063/ [6922064] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/rados_workunit_loadgen_mix}} time: 00:26:04 info: http://pulpito.ceph.com/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922064/ [6922065] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/rbd_api_tests}} time: 00:30:58 info: http://pulpito.ceph.com/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922065/ [6922067] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/rbd_fsx}} time: 00:19:53 info: http://pulpito.ceph.com/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922067/ [6922068] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/rbd_python_api_tests}} time: 00:25:25 info: http://pulpito.ceph.com/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922068/ [6922069] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/rbd_workunit_suites_iozone}} time: 00:21:14 info: http://pulpito.ceph.com/teuthology-2022-07-10_05:00:04-smoke-main-distro-default-smithi/6922069/