2022-07-19T05:01:32.282 INFO:root:teuthology version: 1.1.1.dev383+g89d874d5 2022-07-19T05:01:32.286 INFO:teuthology.results:seed: '8643' 2022-07-19T05:41:56.797 INFO:root:teuthology version: 1.1.1.dev383+g89d874d5 2022-07-19T05:41:58.716 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-07-19T08:59:18.760 INFO:teuthology.results:Tests finished! gathering results... 2022-07-19T08:59:21.033 INFO:scrape:Found 24 jobs 2022-07-19T08:59:23.308 INFO:scrape:Found 3 distinct failure reasons 2022-07-19T08:59:23.309 INFO:scrape:Failure: {'smithi083.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi174.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi055.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} 2022-07-19T08:59:23.310 INFO:scrape:4 jobs: ['6937683', '6937674', '6937676', '6937673'] 2022-07-19T08:59:23.310 INFO:scrape:suites intersection: ['objectstore/bluestore-bitmap', 'openstack}', 'smoke/basic/{clusters/{fixed-3-cephfs', 'supported-random-distro$/{rhel_8}', 'tasks/{0-install'] 2022-07-19T08:59:23.311 INFO:scrape:suites union: ['objectstore/bluestore-bitmap', 'openstack}', 'smoke/basic/{clusters/{fixed-3-cephfs', 'supported-random-distro$/{rhel_8}', 'tasks/{0-install', 'test/rados_cache_snaps}}', 'test/rados_cls_all}}', 'test/rados_python}}', 'test/rgw_ec_s3tests}}'] 2022-07-19T08:59:23.312 INFO:scrape: 2022-07-19T08:59:23.312 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rados/test.sh 2022-07-19T08:59:23.313 INFO:scrape:1 jobs: ['6937670'] 2022-07-19T08:59:23.313 INFO:scrape:suites: ['objectstore/bluestore-bitmap', 'openstack}', 'smoke/basic/{clusters/{fixed-3-cephfs', 'supported-random-distro$/{centos_8}', 'tasks/{0-install', 'test/mon_thrash}}'] 2022-07-19T08:59:23.314 INFO:scrape: 2022-07-19T08:59:23.315 INFO:scrape:Failure: Command failed (s3 tests against rgw) on smithi165 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt /home/ubuntu/cephtest/s3-tests-client.0/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests-client.0 -v -a '!fails_on_rgw,!lifecycle_expiration,!fails_strict_rfc2616,!test_of_sts,!webidentity_test,!fails_with_subdomain,!sse-s3'" 2022-07-19T08:59:23.316 INFO:scrape:1 jobs: ['6937684'] 2022-07-19T08:59:23.318 INFO:scrape:suites: ['objectstore/bluestore-bitmap', 'openstack}', 'smoke/basic/{clusters/{fixed-3-cephfs', 'supported-random-distro$/{ubuntu_latest}', 'tasks/{0-install', 'test/rgw_s3tests}}'] 2022-07-19T08:59:23.318 INFO:scrape: 2022-07-19T08:59:23.320 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/ failed: 2 dead: 4 running: 0 waiting: 0 queued: 0 passed: 18 Fail ================================================================= [6937670] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/mon_thrash}} ----------------------------------------------------------------- time: 03:32:09 info: http://pulpito.ceph.com/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937670/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937670/ sentry: https://sentry.ceph.com/organizations/ceph/?query=828fec77c2e14942bad4f324d2154298 Command failed (workunit test rados/test.sh) on smithi156 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=553faca538f3a7a0ae17e649809d9734c1edd35b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' [6937684] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rgw_s3tests}} ----------------------------------------------------------------- time: 00:26:30 info: http://pulpito.ceph.com/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937684/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937684/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b9fe432ef6c9402e88a811477daeac57 Command failed (s3 tests against rgw) on smithi165 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt /home/ubuntu/cephtest/s3-tests-client.0/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests-client.0 -v -a '!fails_on_rgw,!lifecycle_exp iration,!fails_strict_rfc2616,!test_of_sts,!webidentity_test,!fails_with_su bdomain,!sse-s3'" Dead ================================================================= [6937673] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/rados_cache_snaps}} ----------------------------------------------------------------- time: 00:13:26 info: http://pulpito.ceph.com/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937673/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937673/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4545f4b4f99d47748d0ebafda6554671 {'smithi130.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi045.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi146.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} [6937674] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/rados_cls_all}} ----------------------------------------------------------------- time: 00:12:20 info: http://pulpito.ceph.com/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937674/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937674/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ed639913b1c542378ff74a4b72d7cdb9 {'smithi175.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi125.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi050.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} [6937676] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/rados_python}} ----------------------------------------------------------------- time: 00:12:27 info: http://pulpito.ceph.com/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937676/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937676/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fc5c6604223241739f853c0c78f45e7e {'smithi161.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi123.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi057.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} [6937683] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/rgw_ec_s3tests}} ----------------------------------------------------------------- time: 00:12:40 info: http://pulpito.ceph.com/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937683/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937683/ sentry: https://sentry.ceph.com/organizations/ceph/?query=68a891f0e22446948d6bf39172969bd3 {'smithi083.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi174.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi055.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} Pass ================================================================= [6937661] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/cfuse_workunit_suites_blogbench}} time: 00:20:13 info: http://pulpito.ceph.com/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937661/ [6937662] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/cfuse_workunit_suites_fsstress}} time: 00:15:11 info: http://pulpito.ceph.com/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937662/ [6937663] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/cfuse_workunit_suites_iozone}} time: 00:18:00 info: http://pulpito.ceph.com/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937663/ [6937664] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/cfuse_workunit_suites_pjd}} time: 00:14:27 info: http://pulpito.ceph.com/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937664/ [6937665] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/kclient_workunit_direct_io}} time: 00:14:19 info: http://pulpito.ceph.com/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937665/ [6937666] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/kclient_workunit_suites_dbench}} time: 01:16:20 info: http://pulpito.ceph.com/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937666/ [6937667] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/kclient_workunit_suites_fsstress}} time: 00:13:13 info: http://pulpito.ceph.com/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937667/ [6937668] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/kclient_workunit_suites_pjd}} time: 00:13:48 info: http://pulpito.ceph.com/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937668/ [6937669] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} tasks/{0-install test/libcephfs_interface_tests}} time: 00:20:32 info: http://pulpito.ceph.com/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937669/ [6937671] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rados_api_tests}} time: 00:28:36 info: http://pulpito.ceph.com/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937671/ [6937672] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rados_bench}} time: 02:48:54 info: http://pulpito.ceph.com/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937672/ [6937675] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/rados_ec_snaps}} time: 00:28:11 info: http://pulpito.ceph.com/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937675/ [6937677] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/rados_workunit_loadgen_mix}} time: 00:25:49 info: http://pulpito.ceph.com/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937677/ [6937678] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/rbd_api_tests}} time: 00:30:16 info: http://pulpito.ceph.com/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937678/ [6937679] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rbd_cli_import_export}} time: 00:13:47 info: http://pulpito.ceph.com/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937679/ [6937680] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{centos_8} tasks/{0-install test/rbd_fsx}} time: 00:18:25 info: http://pulpito.ceph.com/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937680/ [6937681] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rbd_python_api_tests}} time: 00:24:33 info: http://pulpito.ceph.com/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937681/ [6937682] smoke/basic/{clusters/{fixed-3-cephfs openstack} objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} tasks/{0-install test/rbd_workunit_suites_iozone}} time: 00:18:29 info: http://pulpito.ceph.com/teuthology-2022-07-19_05:00:03-smoke-main-distro-default-smithi/6937682/