2023-10-17T17:45:05.986 INFO:root:teuthology version: 0.0.1.dev199+gada4f44 2023-10-17T17:45:05.987 INFO:teuthology.results:subset: '111/12000' 2023-10-17T17:45:05.987 INFO:teuthology.results:seed: '993' 2023-10-17T19:14:17.132 INFO:root:teuthology version: 0.0.1.dev199+gada4f44 2023-10-17T19:14:17.161 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-10-17T21:47:28.846 INFO:teuthology.results:Tests finished! gathering results... 2023-10-17T21:47:31.963 INFO:scrape:Found 41 jobs 2023-10-17T21:47:39.802 INFO:scrape:Found 5 distinct failure reasons 2023-10-17T21:47:39.803 INFO:scrape:Failure: Command failed (ragweed tests against rgw) on smithi144 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/ragweed.client.0 && RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg tox --sitepackages -- -v -m 'not fails_on_rgw'" 2023-10-17T21:47:39.804 INFO:scrape:10 jobs: ['7430165', '7430147', '7430170', '7430172', '7430146', '7430164', '7430141', '7430140', '7430171', '7430139'] 2023-10-17T21:47:39.804 INFO:scrape:suites intersection: ['frontend/beast', 'objectstore/bluestore-bitmap'] 2023-10-17T21:47:39.805 INFO:scrape:suites union: ['2-setup', '3-upgrade-sequence/osds-then-rgws', '3-upgrade-sequence/rgws-then-osds', 'cluster', 'clusters/fixed-2', 'datacache/no_datacache', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'inline-data$/{on}', 'install', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'overrides}', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw/multifs/{clusters/fixed-2', 'rgw/upgrade/{1-install/pacific/{distro$/{ubuntu_20.04}', 'rgw/upgrade/{1-install/quincy/{distro$/{centos_latest}', 'rgw/upgrade/{1-install/quincy/{distro$/{ubuntu_20.04}', 'rgw/verify/{0-install', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{default}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'striping$/{stripe-greater-than-chunk}', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/rgw_ragweed', 'tasks/{cls', 'ubuntu_latest}', 'validater/lockdep}', 'validater/valgrind}', 'versioning}'] 2023-10-17T21:47:39.805 INFO:scrape: 2023-10-17T21:47:39.806 INFO:scrape:Failure: "2023-10-17T19:06:18.342973+0000 mon.a (mon.0) 106 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log 2023-10-17T21:47:39.806 INFO:scrape:5 jobs: ['7430163', '7430166', '7430158', '7430161', '7430144'] 2023-10-17T21:47:39.807 INFO:scrape:suites intersection: [] 2023-10-17T21:47:39.807 INFO:scrape:suites union: ['cluster', 'frontend/beast', 'http', 'ignore-pg-availability', 'objectstore/bluestore-bitmap', 'overrides', 'rgw/multifs/{clusters/fixed-2', 'rgw/service-token/{clusters/fixed-1', 'rgw/tempest/{clusters/fixed-1', 'rgw/tools/{centos_latest', 'rgw/website/{clusters/fixed-2', 'rgw_pool_type/ec', 's3tests-branch', 'tasks/rgw_s3tests', 'tasks/rgw_tempest', 'tasks/s3tests-website', 'tasks/service-token', 'tasks}', 'ubuntu_latest}'] 2023-10-17T21:47:39.808 INFO:scrape: 2023-10-17T21:47:39.808 INFO:scrape:Failure: rgw multisite test failures 2023-10-17T21:47:39.809 INFO:scrape:2 jobs: ['7430137', '7430156'] 2023-10-17T21:47:39.809 INFO:scrape:suites intersection: ['frontend/beast', 'ignore-pg-availability', 'notify', 'omap_limits', 'overrides', 'rgw/multisite/{clusters', 'supported-random-distro$/{centos_8}', 'tasks/test_multi}'] 2023-10-17T21:47:39.809 INFO:scrape:suites union: ['frontend/beast', 'ignore-pg-availability', 'notify', 'omap_limits', 'overrides', 'realms/three-zones', 'realms/two-zonegroup', 'rgw/multisite/{clusters', 'supported-random-distro$/{centos_8}', 'tasks/test_multi}'] 2023-10-17T21:47:39.810 INFO:scrape: 2023-10-17T21:47:39.810 INFO:scrape:Failure: Command failed on smithi155 with status 100: 'DEBIAN_FRONTEND=noninteractive sudo -E apt-get -y --force-yes install python-dev' 2023-10-17T21:47:39.811 INFO:scrape:1 jobs: ['7430168'] 2023-10-17T21:47:39.811 INFO:scrape:suites: ['1-ceph-install/install', '2-kms/kmip', '3-rgw/rgw', '4-tests/{s3tests}', 'ignore-pg-availability', 'rgw/crypt/{0-cluster/fixed-1', 's3tests-branch', 'ubuntu_latest}'] 2023-10-17T21:47:39.812 INFO:scrape: 2023-10-17T21:47:39.812 INFO:scrape:Failure: Command failed on smithi107 with status 127: 'cd /home/ubuntu/cephtest/keycloak-11.0.0 && cd bin && ./jboss-cli.sh --file=adapter-elytron-install-offline.cli' 2023-10-17T21:47:39.812 INFO:scrape:1 jobs: ['7430160'] 2023-10-17T21:47:39.813 INFO:scrape:suites: ['first', 'ignore-pg-availability', 'objectstore', 'overrides', 'pool-type', 'rgw/sts/{cluster', 'rgw_frontend/beast', 's3tests-branch', 'ststests}}', 'supported-random-distro$/{centos_latest}', 'tasks/{0-install'] 2023-10-17T21:47:39.814 INFO:scrape: 2023-10-17T21:47:39.815 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/ failed: 19 dead: 0 running: 0 waiting: 0 queued: 0 passed: 22 Fail ================================================================= [7430137] rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zonegroup supported-random-distro$/{centos_8} tasks/test_multi} ----------------------------------------------------------------- time: 01:56:01 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430137/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430137/ sentry: https://sentry.ceph.com/organizations/ceph/?query=60920426ddef413bb577b22d1275130e rgw multisite test failures [7430139] rgw/upgrade/{1-install/quincy/{distro$/{centos_latest} install overrides} 2-setup 3-upgrade-sequence/osds-then-rgws cluster frontend/beast objectstore/bluestore-bitmap overrides} ----------------------------------------------------------------- time: 00:14:07 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430139/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430139/ sentry: https://sentry.ceph.com/organizations/ceph/?query=54e177673e7142ee808d9c2dfc4e3fea Command failed (ragweed tests against rgw) on smithi092 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/ragweed.client.1 && RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.1.conf RAGWEED_STAGES=check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg tox --sitepackages -- -v -m 'not fails_on_rgw'" [7430140] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{rhel_8} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep} ----------------------------------------------------------------- time: 00:23:43 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430140/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430140/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2fd13076728a476b8e59641ecb48d839 Command failed (ragweed tests against rgw) on smithi029 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/ragweed.client.0 && RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg tox --sitepackages -- -v -m 'not fails_on_rgw'" [7430141] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_ragweed ubuntu_latest} ----------------------------------------------------------------- time: 00:12:00 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430141/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430141/ sentry: https://sentry.ceph.com/organizations/ceph/?query=86c77dffb0864237ae798cb086cdd85e Command failed (ragweed tests against rgw) on smithi022 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/ragweed.client.0 && RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg tox --sitepackages -- -v -m 'not fails_on_rgw'" [7430144] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_s3tests ubuntu_latest} ----------------------------------------------------------------- time: 00:20:24 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430144/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430144/ "2023-10-17T18:40:00.000173+0000 mon.a (mon.0) 162 : cluster [WRN] overall HEALTH_WARN 1 pool(s) do not have an application enabled" in cluster log [7430146] rgw/upgrade/{1-install/pacific/{distro$/{ubuntu_20.04} install overrides} 2-setup 3-upgrade-sequence/rgws-then-osds cluster frontend/beast objectstore/bluestore-bitmap overrides} ----------------------------------------------------------------- time: 00:13:08 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430146/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430146/ sentry: https://sentry.ceph.com/organizations/ceph/?query=942fdeba1c3f4e04b5c668f0cf5bc5d3 Command failed (ragweed tests against rgw) on smithi124 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/ragweed.client.0 && RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg tox --sitepackages -- -v -m 'not fails_on_rgw'" [7430147] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{rhel_8} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} ----------------------------------------------------------------- time: 00:27:43 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430147/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430147/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c14d65e0cef74250afebd374e930b2d7 Command failed (ragweed tests against rgw) on smithi017 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/ragweed.client.0 && RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg tox --sitepackages -- -v -m 'not fails_on_rgw'" [7430156] rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zones supported-random-distro$/{centos_8} tasks/test_multi} ----------------------------------------------------------------- time: 02:52:32 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430156/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430156/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b0af3c43c53e49d6b87bf501e47d8cd8 rgw multisite test failures [7430158] rgw/service-token/{clusters/fixed-1 frontend/beast overrides tasks/service-token ubuntu_latest} ----------------------------------------------------------------- time: 00:17:54 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430158/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430158/ "2023-10-17T18:55:46.111885+0000 mon.a (mon.0) 108 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log [7430160] rgw/sts/{cluster ignore-pg-availability objectstore overrides pool-type rgw_frontend/beast s3tests-branch supported-random-distro$/{centos_latest} tasks/{0-install first ststests}} ----------------------------------------------------------------- time: 00:09:31 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430160/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430160/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b3d4b48dd0a74715909854dc33849853 Command failed on smithi107 with status 127: 'cd /home/ubuntu/cephtest/keycloak-11.0.0 && cd bin && ./jboss-cli.sh --file=adapter-elytron-install-offline.cli' [7430161] rgw/tempest/{clusters/fixed-1 frontend/beast overrides tasks/rgw_tempest ubuntu_latest} ----------------------------------------------------------------- time: 00:25:00 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430161/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430161/ "2023-10-17T19:07:29.070501+0000 mon.a (mon.0) 108 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log [7430163] rgw/tools/{centos_latest cluster tasks} ----------------------------------------------------------------- time: 02:48:49 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430163/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430163/ "2023-10-17T19:06:18.342973+0000 mon.a (mon.0) 106 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log [7430164] rgw/upgrade/{1-install/pacific/{distro$/{ubuntu_20.04} install overrides} 2-setup 3-upgrade-sequence/osds-then-rgws cluster frontend/beast objectstore/bluestore-bitmap overrides} ----------------------------------------------------------------- time: 00:13:07 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430164/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430164/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e55b2d01623b467ea5ca43eb4e5e78c5 Command failed (ragweed tests against rgw) on smithi005 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/ragweed.client.0 && RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg tox --sitepackages -- -v -m 'not fails_on_rgw'" [7430165] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_8} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep} ----------------------------------------------------------------- time: 00:22:30 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430165/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430165/ sentry: https://sentry.ceph.com/organizations/ceph/?query=48e917c851174806998d6986e244a8af Command failed (ragweed tests against rgw) on smithi144 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/ragweed.client.0 && RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg tox --sitepackages -- -v -m 'not fails_on_rgw'" [7430166] rgw/website/{clusters/fixed-2 frontend/beast http overrides s3tests-branch tasks/s3tests-website ubuntu_latest} ----------------------------------------------------------------- time: 00:23:21 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430166/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430166/ "2023-10-17T19:13:14.921033+0000 mon.a (mon.0) 176 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log [7430168] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/kmip 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch ubuntu_latest} ----------------------------------------------------------------- time: 00:10:33 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430168/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430168/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d4b541d2637240cca5a27562f5a6095a Command failed on smithi155 with status 100: 'DEBIAN_FRONTEND=noninteractive sudo -E apt-get -y --force-yes install python-dev' [7430170] rgw/upgrade/{1-install/quincy/{distro$/{ubuntu_20.04} install overrides} 2-setup 3-upgrade-sequence/rgws-then-osds cluster frontend/beast objectstore/bluestore-bitmap overrides} ----------------------------------------------------------------- time: 00:12:33 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430170/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430170/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d11f479ab553450da94375f286dbae25 Command failed (ragweed tests against rgw) on smithi032 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/ragweed.client.0 && RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg tox --sitepackages -- -v -m 'not fails_on_rgw'" [7430171] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} ----------------------------------------------------------------- time: 00:26:47 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430171/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430171/ sentry: https://sentry.ceph.com/organizations/ceph/?query=77bd34c383504d9f8e4872c769435d4e Command failed (ragweed tests against rgw) on smithi110 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/ragweed.client.0 && RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg tox --sitepackages -- -v -m 'not fails_on_rgw'" [7430172] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_ragweed ubuntu_latest} ----------------------------------------------------------------- time: 00:12:08 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430172/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430172/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f6c1b77b53324ca3a668054e2a80ce8c Command failed (ragweed tests against rgw) on smithi017 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/ragweed.client.0 && RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg tox --sitepackages -- -v -m 'not fails_on_rgw'" Pass ================================================================= [7430136] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/v32 overrides s3a-hadoop} time: 00:26:46 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430136/ [7430138] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_s3tests} time: 00:23:39 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430138/ [7430142] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_old 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch ubuntu_latest} time: 00:19:36 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430142/ [7430143] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_20.04}} time: 00:11:59 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430143/ [7430145] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota} time: 00:22:00 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430145/ [7430148] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_transit 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch ubuntu_latest} time: 00:19:35 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430148/ [7430149] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_user_quota ubuntu_latest} time: 00:15:38 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430149/ [7430150] rgw/cloud-transition/{cluster overrides s3tests-branch supported-random-distro$/{rhel_8} tasks/cloud_transition_s3tests} time: 00:35:15 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430150/ [7430151] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/barbican 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch ubuntu_latest} time: 00:28:43 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430151/ [7430152] rgw/dbstore/{cluster overrides s3tests-branch supported-random-distro$/{centos_8} tasks/rgw_s3tests} time: 00:19:16 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430152/ [7430153] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/default overrides s3a-hadoop} time: 00:20:04 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430153/ [7430154] rgw/lifecycle/{cluster overrides s3tests-branch supported-random-distro$/{centos_8} tasks/rgw_s3tests} time: 00:27:29 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430154/ [7430155] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_bucket_quota ubuntu_latest} time: 00:17:12 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430155/ [7430157] rgw/notifications/{beast bluestore-bitmap fixed-2 overrides supported-all-distro$/{centos_8} tasks/{0-install test_amqp test_kafka test_others}} time: 00:38:20 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430157/ [7430159] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{ubuntu_latest}} time: 00:13:58 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430159/ [7430162] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota} time: 00:17:26 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430162/ [7430167] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_multipart_upload ubuntu_latest} time: 00:16:12 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430167/ [7430169] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_multipart_upload} time: 00:18:32 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430169/ [7430173] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/testing 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch ubuntu_latest} time: 00:19:11 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430173/ [7430175] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{ubuntu_latest}} time: 00:13:49 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430175/ [7430177] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_s3tests ubuntu_latest} time: 00:20:25 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430177/ [7430179] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_kv 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch ubuntu_latest} time: 00:19:26 info: http://pulpito.ceph.com/yuriw-2023-10-16_21:55:28-rgw-wip-yuri-testing-2023-10-16-1247-reef-distro-default-smithi/7430179/