2022-12-14T17:19:04.624 INFO:root:teuthology version: 0.0.1.dev18+g19d18db 2022-12-14T17:19:04.624 INFO:teuthology.results:subset: '111/120000' 2022-12-14T17:19:04.624 INFO:teuthology.results:seed: '8817' 2022-12-14T17:45:31.518 INFO:root:teuthology version: 0.0.1.dev18+g19d18db 2022-12-14T17:45:31.539 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-12-14T22:33:18.695 INFO:teuthology.results:Tests finished! gathering results... 2022-12-14T22:33:27.349 INFO:scrape:Found 43 jobs 2022-12-14T22:33:27.355 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116531/teuthology.log 2022-12-14T22:33:30.654 INFO:scrape:Found 7 distinct failure reasons 2022-12-14T22:33:30.655 INFO:scrape:Dead: None 2022-12-14T22:33:30.655 INFO:scrape:3 jobs: ['7116531', '7116562', '7116548'] 2022-12-14T22:33:30.655 INFO:scrape: 2022-12-14T22:33:30.655 INFO:scrape:Failure: Command failed on smithi133 with status 1: 'cd /home/ubuntu/cephtest/tempest && /home/ubuntu/cephtest/tox-venv/bin/tox -e venv --notest' 2022-12-14T22:33:30.656 INFO:scrape:1 jobs: ['7116529'] 2022-12-14T22:33:30.656 INFO:scrape:suites: ['frontend/beast', 'overrides', 'rgw/tempest/{clusters/fixed-1', 'tasks/rgw_tempest', 'ubuntu_latest}'] 2022-12-14T22:33:30.656 INFO:scrape: 2022-12-14T22:33:30.656 INFO:scrape:Failure: Stale jobs detected, aborting. 2022-12-14T22:33:30.656 INFO:scrape:5 jobs: ['7116563', '7116550', '7116534', '7116552', '7116533'] 2022-12-14T22:33:30.656 INFO:scrape:suites intersection: [] 2022-12-14T22:33:30.656 INFO:scrape:suites union: ['clusters/fixed-2', 'first', 'frontend/beast', 'frontend/civetweb', 'http', 'ignore-pg-availability', 'install', 'objectstore', 'objectstore/filestore-xfs', 'overrides', 'pool-type', 'rgw/multifs/{clusters/fixed-2', 'rgw/sts/{cluster', 'rgw/thrash/{civetweb', 'rgw/website/{clusters/fixed-2', 'rgw_frontend/civetweb', 'rgw_pool_type/replicated', 'ststests}}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/rgw_multipart_upload}', 'tasks/s3tests-website', 'tasks/{0-install', 'thrasher/default', 'thrashosds-health', 'ubuntu_latest}', 'workload/rgw_bucket_quota}', 'workload/rgw_multipart_upload}'] 2022-12-14T22:33:30.657 INFO:scrape: 2022-12-14T22:33:30.657 INFO:scrape:Failure: rgw multisite, rgw multisite pubsub, test failures 2022-12-14T22:33:30.657 INFO:scrape:2 jobs: ['7116526', '7116559'] 2022-12-14T22:33:30.657 INFO:scrape:suites intersection: ['frontend/beast', 'ignore-pg-availability', 'omap_limits', 'overrides', 'rgw/multisite/{clusters', 'tasks/test_multi', 'valgrind}'] 2022-12-14T22:33:30.657 INFO:scrape:suites union: ['frontend/beast', 'ignore-pg-availability', 'omap_limits', 'overrides', 'realms/three-zone-plus-pubsub', 'realms/two-zonegroup', 'rgw/multisite/{clusters', 'tasks/test_multi', 'valgrind}'] 2022-12-14T22:33:30.657 INFO:scrape: 2022-12-14T22:33:30.657 INFO:scrape:Failure: {'smithi136.front.sepia.ceph.com': {'changed': False, 'msg': 'All items completed', 'results': [{'_ansible_item_label': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': False, 'err': " /dev/vg_nvme: already exists in filesystem\n Run `vgcreate --help' for more information.\n", 'failed': True, 'invocation': {'module_args': {'force': False, 'pesize': '4', 'pv_options': '', 'pvresize': False, 'pvs': ['/dev/nvme0n1'], 'state': 'present', 'vg': 'vg_nvme', 'vg_options': ''}}, 'item': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, 'msg': "Creating volume group 'vg_nvme' failed", 'rc': 3}]}} 2022-12-14T22:33:30.657 INFO:scrape:9 jobs: ['7116560', '7116561', '7116536', '7116535', '7116564', '7116549', '7116532', '7116537', '7116551'] 2022-12-14T22:33:30.658 INFO:scrape:suites intersection: [] 2022-12-14T22:33:30.658 INFO:scrape:suites union: ['1-ceph-install/install', '2-kms/kmip', '3-rgw/rgw', '4-tests/{s3tests}', 'cls', 'clusters/fixed-2', 'frontend/beast', 'frontend/civetweb', 'ignore-pg-availability', 'ignore-pg-availability}', 'install', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'overrides', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw/crypt/{0-cluster/fixed-1', 'rgw/multifs/{clusters/fixed-2', 'rgw/singleton/{all/radosgw-admin', 'rgw/tempest/{clusters/fixed-1', 'rgw/thrash/{civetweb', 'rgw/verify/{centos_latest', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 's3tests-java', 's3tests}', 'sharding$/{default}', 'striping$/{stripe-equals-chunk}', 'striping$/{stripe-greater-than-chunk}', 'supported-random-distro$/{rhel_8}}', 'supported-random-distro$/{ubuntu_latest}}', 'tasks/rgw_ragweed}', 'tasks/rgw_tempest', 'tasks/{0-install', 'thrasher/default', 'thrashosds-health', 'ubuntu_latest}', 'validater/lockdep}', 'validater/valgrind}', 'workload/rgw_s3tests}'] 2022-12-14T22:33:30.658 INFO:scrape: 2022-12-14T22:33:30.658 INFO:scrape:Failure: Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri6-testing-2022-12-14-0726-pacific 2022-12-14T22:33:30.658 INFO:scrape:2 jobs: ['7116540', '7116527'] 2022-12-14T22:33:30.658 INFO:scrape:suites: ['frontend/civetweb', 'objectstore/bluestore-bitmap', 'overrides', 'rgw/singleton/{all/radosgw-admin', 'rgw_pool_type/ec-profile', 'supported-random-distro$/{centos_8}}'] 2022-12-14T22:33:30.658 INFO:scrape: 2022-12-14T22:33:30.659 INFO:scrape:Failure: Cannot create secret 2022-12-14T22:33:30.659 INFO:scrape:1 jobs: ['7116523'] 2022-12-14T22:33:30.659 INFO:scrape:suites: ['1-ceph-install/install', '2-kms/barbican', '3-rgw/rgw', '4-tests/{s3tests}', 'ignore-pg-availability}', 'rgw/crypt/{0-cluster/fixed-1'] 2022-12-14T22:33:30.659 INFO:scrape: 2022-12-14T22:33:30.660 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/ failed: 11 dead: 12 running: 0 waiting: 0 queued: 0 passed: 20 Fail ================================================================= [7116523] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/barbican 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability} ----------------------------------------------------------------- time: 00:17:06 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116523/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116523/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1cabf43fdcae4e268fa015f7e0fef5bf Cannot create secret [7116526] rgw/multisite/{clusters frontend/beast ignore-pg-availability omap_limits overrides realms/three-zone-plus-pubsub tasks/test_multi valgrind} ----------------------------------------------------------------- time: 04:57:29 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116526/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116526/ sentry: https://sentry.ceph.com/organizations/ceph/?query=97aa649dc9964169bc6e4f4c8d9e51f6 rgw multisite, rgw multisite pubsub, test failures [7116527] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:08:29 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116527/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116527/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8164c9cf812449adb477d4b9e4fecd27 Failed to fetch package version from https://shaman.ceph.com/api/search/?st atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=92f 5cf4fe01cdafe9619063a0620f0401196377a [7116529] rgw/tempest/{clusters/fixed-1 frontend/beast overrides tasks/rgw_tempest ubuntu_latest} ----------------------------------------------------------------- time: 00:23:15 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116529/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116529/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a1e9d1d390cd4b9babd4be3849ac4fdf Command failed on smithi133 with status 1: 'cd /home/ubuntu/cephtest/tempest && /home/ubuntu/cephtest/tox-venv/bin/tox -e venv --notest' [7116533] rgw/website/{clusters/fixed-2 frontend/beast http overrides tasks/s3tests-website ubuntu_latest} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116533/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116533/ sentry: https://sentry.ceph.com/organizations/ceph/?query=778593c01a4d420394fc805a9bf78406 Stale jobs detected, aborting. [7116534] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_multipart_upload} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116534/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116534/ sentry: https://sentry.ceph.com/organizations/ceph/?query=006d80fb23a44f4f8f5f4f1aafcf3c0f Stale jobs detected, aborting. [7116540] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116540/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116540/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d3f3bd7034084333832fc49187c1c80a Failed to fetch package version from https://shaman.ceph.com/api/search/?st atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip- yuri6-testing-2022-12-14-0726-pacific [7116550] rgw/sts/{cluster ignore-pg-availability objectstore overrides pool-type rgw_frontend/civetweb supported-random-distro$/{ubuntu_latest} tasks/{0-install first ststests}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116550/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116550/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0aa2da7299a6445eb88c6f99fc933760 Stale jobs detected, aborting. [7116552] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_bucket_quota} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116552/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116552/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c70ec01a68a9474fabc9601068d3ce28 Stale jobs detected, aborting. [7116559] rgw/multisite/{clusters frontend/beast ignore-pg-availability omap_limits overrides realms/two-zonegroup tasks/test_multi valgrind} ----------------------------------------------------------------- time: 00:40:41 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116559/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116559/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5b92a29dcae34ef499d3002f97c74660 rgw multisite, test failures [7116563] rgw/multifs/{clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/replicated tasks/rgw_multipart_upload} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116563/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116563/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7258bba64e324dea8e3ba36d56541f06 Stale jobs detected, aborting. Dead ================================================================= [7116531] rgw/tools/{centos_latest cluster tasks} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116531/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116531/ Error reimaging machines: Failed to power on smithi089 [7116532] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:07:42 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116532/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116532/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f8e9ba8751d3466c8feb5878c18521c8 {'smithi137.front.sepia.ceph.com': {'changed': False, 'msg': 'All items completed', 'results': [{'_ansible_item_label': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': False, 'err': " /dev/vg_nvme: already exists in filesystem\n Run `vgcreate --help' for more information.\n", 'failed': True, 'invocation': {'module_args': {'force': False, 'pesize': '4', 'pv_options': '', 'pvresize': False, 'pvs': ['/dev/nvme0n1'], 'state': 'present', 'vg': 'vg_nvme', 'vg_options': ''}}, 'item': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, 'msg': "Creating volume group 'vg_nvme' failed", 'rc': 3}]}} [7116535] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/kmip 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability} ----------------------------------------------------------------- time: 00:02:26 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116535/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116535/ sentry: https://sentry.ceph.com/organizations/ceph/?query=40c2e8e020644c75b1c91b676f2435aa {'smithi130.front.sepia.ceph.com': {'changed': False, 'msg': 'All items completed', 'results': [{'_ansible_item_label': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': False, 'err': " /dev/vg_nvme: already exists in filesystem\n Run `vgcreate --help' for more information.\n", 'failed': True, 'invocation': {'module_args': {'force': False, 'pesize': '4', 'pv_options': '', 'pvresize': False, 'pvs': ['/dev/nvme0n1'], 'state': 'present', 'vg': 'vg_nvme', 'vg_options': ''}}, 'item': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, 'msg': "Creating volume group 'vg_nvme' failed", 'rc': 3}]}} [7116536] rgw/multifs/{clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec tasks/rgw_ragweed} ----------------------------------------------------------------- time: 00:04:49 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116536/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116536/ sentry: https://sentry.ceph.com/organizations/ceph/?query=45fdc7b1e2b543c9a80b87c1553f027c {'smithi142.front.sepia.ceph.com': {'changed': False, 'msg': 'All items completed', 'results': [{'_ansible_item_label': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': False, 'err': " /dev/vg_nvme: already exists in filesystem\n Run `vgcreate --help' for more information.\n", 'failed': True, 'invocation': {'module_args': {'force': False, 'pesize': '4', 'pv_options': '', 'pvresize': False, 'pvs': ['/dev/nvme0n1'], 'state': 'present', 'vg': 'vg_nvme', 'vg_options': ''}}, 'item': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, 'msg': "Creating volume group 'vg_nvme' failed", 'rc': 3}]}} [7116537] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:04:46 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116537/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116537/ sentry: https://sentry.ceph.com/organizations/ceph/?query=088e79dec6ea481e8c61d7a71cded693 {'smithi181.front.sepia.ceph.com': {'changed': False, 'msg': 'All items completed', 'results': [{'_ansible_item_label': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': False, 'err': " /dev/vg_nvme: already exists in filesystem\n Run `vgcreate --help' for more information.\n", 'failed': True, 'invocation': {'module_args': {'force': False, 'pesize': '4', 'pv_options': '', 'pvresize': False, 'pvs': ['/dev/nvme0n1'], 'state': 'present', 'vg': 'vg_nvme', 'vg_options': ''}}, 'item': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, 'msg': "Creating volume group 'vg_nvme' failed", 'rc': 3}]}, 'smithi003.front.sepia.ceph.com': {'changed': False, 'msg': 'All items completed', 'results': [{'_ansible_item_label': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': False, 'err': " /dev/vg_nvme: already exists in filesystem\n Run `vgcreate --help' for more information.\n", 'failed': True, 'invocation': {'module_args': {'force': False, 'pesize': '4', 'pv_options': '', 'pvresize': False, 'pvs': ['/dev/nvme0n1'], 'state': 'present', 'vg': 'vg_nvme', 'vg_options': ''}}, 'item': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, 'msg': "Creating volume group 'vg_nvme' failed", 'rc': 3}]}} [7116548] rgw/multisite/{clusters frontend/civetweb ignore-pg-availability omap_limits overrides realms/three-zone-plus-pubsub tasks/test_multi valgrind} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116548/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116548/ Error reimaging machines: Failed to power on smithi132 [7116549] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec-profile supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:04:24 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116549/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116549/ sentry: https://sentry.ceph.com/organizations/ceph/?query=edc4448e19f442d1aacf18e646070f08 {'smithi119.front.sepia.ceph.com': {'changed': False, 'msg': 'All items completed', 'results': [{'_ansible_item_label': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': False, 'err': " /dev/vg_nvme: already exists in filesystem\n Run `vgcreate --help' for more information.\n", 'failed': True, 'invocation': {'module_args': {'force': False, 'pesize': '4', 'pv_options': '', 'pvresize': False, 'pvs': ['/dev/nvme0n1'], 'state': 'present', 'vg': 'vg_nvme', 'vg_options': ''}}, 'item': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, 'msg': "Creating volume group 'vg_nvme' failed", 'rc': 3}]}, 'smithi145.front.sepia.ceph.com': {'changed': False, 'msg': 'All items completed', 'results': [{'_ansible_item_label': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': False, 'err': " /dev/vg_nvme: already exists in filesystem\n Run `vgcreate --help' for more information.\n", 'failed': True, 'invocation': {'module_args': {'force': False, 'pesize': '4', 'pv_options': '', 'pvresize': False, 'pvs': ['/dev/nvme0n1'], 'state': 'present', 'vg': 'vg_nvme', 'vg_options': ''}}, 'item': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, 'msg': "Creating volume group 'vg_nvme' failed", 'rc': 3}]}} [7116551] rgw/tempest/{clusters/fixed-1 frontend/civetweb overrides tasks/rgw_tempest ubuntu_latest} ----------------------------------------------------------------- time: 00:04:30 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116551/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116551/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9d879966cfff4e359bb006da57695fe0 {'smithi106.front.sepia.ceph.com': {'changed': False, 'msg': 'All items completed', 'results': [{'_ansible_item_label': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': False, 'err': " /dev/vg_nvme: already exists in filesystem\n Run `vgcreate --help' for more information.\n", 'failed': True, 'invocation': {'module_args': {'force': False, 'pesize': '4', 'pv_options': '', 'pvresize': False, 'pvs': ['/dev/nvme0n1'], 'state': 'present', 'vg': 'vg_nvme', 'vg_options': ''}}, 'item': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, 'msg': "Creating volume group 'vg_nvme' failed", 'rc': 3}]}} [7116560] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_s3tests} ----------------------------------------------------------------- time: 00:04:49 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116560/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116560/ sentry: https://sentry.ceph.com/organizations/ceph/?query=212567725ccc4cde997c5d3b56433226 {'smithi136.front.sepia.ceph.com': {'changed': False, 'msg': 'All items completed', 'results': [{'_ansible_item_label': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': False, 'err': " /dev/vg_nvme: already exists in filesystem\n Run `vgcreate --help' for more information.\n", 'failed': True, 'invocation': {'module_args': {'force': False, 'pesize': '4', 'pv_options': '', 'pvresize': False, 'pvs': ['/dev/nvme0n1'], 'state': 'present', 'vg': 'vg_nvme', 'vg_options': ''}}, 'item': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, 'msg': "Creating volume group 'vg_nvme' failed", 'rc': 3}]}} [7116561] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:02:22 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116561/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116561/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e4d3807d726d41449c1e63488bc4a9dc {'smithi183.front.sepia.ceph.com': {'changed': False, 'msg': 'All items completed', 'results': [{'_ansible_item_label': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': False, 'err': " /dev/vg_nvme: already exists in filesystem\n Run `vgcreate --help' for more information.\n", 'failed': True, 'invocation': {'module_args': {'force': False, 'pesize': '4', 'pv_options': '', 'pvresize': False, 'pvs': ['/dev/nvme0n1'], 'state': 'present', 'vg': 'vg_nvme', 'vg_options': ''}}, 'item': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, 'msg': "Creating volume group 'vg_nvme' failed", 'rc': 3}]}, 'smithi165.front.sepia.ceph.com': {'changed': False, 'msg': 'All items completed', 'results': [{'_ansible_item_label': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': False, 'err': " /dev/vg_nvme: already exists in filesystem\n Run `vgcreate --help' for more information.\n", 'failed': True, 'invocation': {'module_args': {'force': False, 'pesize': '4', 'pv_options': '', 'pvresize': False, 'pvs': ['/dev/nvme0n1'], 'state': 'present', 'vg': 'vg_nvme', 'vg_options': ''}}, 'item': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, 'msg': "Creating volume group 'vg_nvme' failed", 'rc': 3}]}} [7116562] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_transit 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116562/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116562/ Error reimaging machines: Failed to power on smithi181 [7116564] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:03:23 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116564/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116564/ sentry: https://sentry.ceph.com/organizations/ceph/?query=935b44525aab485383421914247ea955 {'smithi026.front.sepia.ceph.com': {'changed': False, 'msg': 'All items completed', 'results': [{'_ansible_item_label': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': False, 'err': " /dev/vg_nvme: already exists in filesystem\n Run `vgcreate --help' for more information.\n", 'failed': True, 'invocation': {'module_args': {'force': False, 'pesize': '4', 'pv_options': '', 'pvresize': False, 'pvs': ['/dev/nvme0n1'], 'state': 'present', 'vg': 'vg_nvme', 'vg_options': ''}}, 'item': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, 'msg': "Creating volume group 'vg_nvme' failed", 'rc': 3}]}} Pass ================================================================= [7116524] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/default overrides s3a-hadoop} time: 00:17:40 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116524/ [7116525] rgw/multifs/{clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile tasks/rgw_multipart_upload} time: 00:10:20 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116525/ [7116528] rgw/sts/{cluster ignore-pg-availability objectstore overrides pool-type rgw_frontend/beast supported-random-distro$/{ubuntu_latest} tasks/{0-install first ststests}} time: 00:28:06 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116528/ [7116530] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health workload/rgw_bucket_quota} time: 00:13:50 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116530/ [7116538] rgw/multisite/{clusters frontend/civetweb ignore-pg-availability omap_limits overrides realms/two-zonegroup tasks/test_multi valgrind} time: 00:33:06 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116538/ [7116539] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health workload/rgw_s3tests} time: 00:18:23 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116539/ [7116541] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/testing 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability} time: 00:12:58 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116541/ [7116542] rgw/multifs/{clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated tasks/rgw_s3tests} time: 00:14:08 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116542/ [7116543] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{centos_8}} time: 00:15:53 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116543/ [7116544] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_user_quota} time: 00:12:56 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116544/ [7116545] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_kv 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability} time: 00:12:51 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116545/ [7116546] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/v32 overrides s3a-hadoop} time: 00:16:37 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116546/ [7116547] rgw/multifs/{clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile tasks/rgw_user_quota} time: 00:09:43 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116547/ [7116553] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} time: 00:29:07 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116553/ [7116554] rgw/website/{clusters/fixed-2 frontend/civetweb http overrides tasks/s3tests-website ubuntu_latest} time: 00:18:03 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116554/ [7116555] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health workload/rgw_multipart_upload} time: 00:13:20 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116555/ [7116556] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_old 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability} time: 00:12:43 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116556/ [7116557] rgw/multifs/{clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec tasks/rgw_bucket_quota} time: 00:11:29 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116557/ [7116558] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{rhel_8}} time: 00:16:02 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116558/ [7116565] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health workload/rgw_user_quota} time: 00:16:27 info: http://pulpito.ceph.com/yuriw-2022-12-14_17:17:08-rgw-wip-yuri6-testing-2022-12-14-0726-pacific-distro-default-smithi/7116565/