2022-06-30T22:36:32.121 INFO:root:teuthology version: 1.1.1.dev379+gf58709e3 2022-06-30T22:36:32.122 INFO:teuthology.results:subset: '111/120000' 2022-06-30T22:36:32.123 INFO:teuthology.results:seed: '9830' 2022-06-30T22:50:33.058 INFO:root:teuthology version: 1.1.1.dev379+gf58709e3 2022-06-30T22:50:33.086 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-07-01T01:48:48.188 INFO:teuthology.results:Tests finished! gathering results... 2022-07-01T01:48:49.245 INFO:scrape:Found 9 jobs 2022-07-01T01:48:52.286 INFO:scrape:Found 2 distinct failure reasons 2022-07-01T01:48:52.287 INFO:scrape:Failure: Command failed on smithi026 with status 123: "sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --" 2022-07-01T01:48:52.287 INFO:scrape:1 jobs: ['6907793'] 2022-07-01T01:48:52.288 INFO:scrape:suites: ['0-roles', '1-start', '2-first-half-tasks/snaps-few-objects', '3-stress-tasks/{radosbench', '4-second-half-tasks/rbd-import-export', 'mon_election/classic}', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}', 'upgrade:pacific-x/stress-split/{0-distro/ubuntu_20.04'] 2022-07-01T01:48:52.288 INFO:scrape: 2022-07-01T01:48:52.289 INFO:scrape:Failure: Command failed (workunit test cls/test_cls_rgw.sh) on smithi083 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=pacific TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh' 2022-07-01T01:48:52.289 INFO:scrape:2 jobs: ['6907795', '6907799'] 2022-07-01T01:48:52.290 INFO:scrape:suites intersection: ['0-start', '1-tasks', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'test_rbd_api', 'test_rbd_python}}', 'upgrade-sequence', 'workload/{ec-rados-default'] 2022-07-01T01:48:52.290 INFO:scrape:suites union: ['0-start', '1-tasks', 'mon_election/classic', 'mon_election/connectivity', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'test_rbd_api', 'test_rbd_python}}', 'upgrade-sequence', 'upgrade:pacific-x/parallel/{0-random-distro$/{centos_8.stream_container_tools_crun}', 'upgrade:pacific-x/parallel/{0-random-distro$/{centos_8.stream_container_tools}', 'workload/{ec-rados-default'] 2022-07-01T01:48:52.291 INFO:scrape: 2022-07-01T01:48:52.293 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2022-06-30_22:36:16-upgrade:pacific-x-wip-yuri3-testing-2022-06-28-1737-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2022-06-30_22:36:16-upgrade:pacific-x-wip-yuri3-testing-2022-06-28-1737-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2022-06-30_22:36:16-upgrade:pacific-x-wip-yuri3-testing-2022-06-28-1737-distro-default-smithi/ failed: 3 dead: 0 running: 0 waiting: 0 queued: 0 passed: 6 Fail ================================================================= [6907793] upgrade:pacific-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} ----------------------------------------------------------------- time: 02:05:45 info: http://pulpito.ceph.com/yuriw-2022-06-30_22:36:16-upgrade:pacific-x-wip-yuri3-testing-2022-06-28-1737-distro-default-smithi/6907793/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-06-30_22:36:16-upgrade:pacific-x-wip-yuri3-testing-2022-06-28-1737-distro-default-smithi/6907793/ Command failed on smithi026 with status 123: "sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if- empty -- gzip --" [6907795] upgrade:pacific-x/parallel/{0-random-distro$/{centos_8.stream_container_tools_crun} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} ----------------------------------------------------------------- time: 00:36:56 info: http://pulpito.ceph.com/yuriw-2022-06-30_22:36:16-upgrade:pacific-x-wip-yuri3-testing-2022-06-28-1737-distro-default-smithi/6907795/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-06-30_22:36:16-upgrade:pacific-x-wip-yuri3-testing-2022-06-28-1737-distro-default-smithi/6907795/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e5bd4e24565a4321b5da58f602e831e4 Command failed (workunit test cls/test_cls_rgw.sh) on smithi083 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=pacific TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh' [6907799] upgrade:pacific-x/parallel/{0-random-distro$/{centos_8.stream_container_tools} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} ----------------------------------------------------------------- time: 00:38:33 info: http://pulpito.ceph.com/yuriw-2022-06-30_22:36:16-upgrade:pacific-x-wip-yuri3-testing-2022-06-28-1737-distro-default-smithi/6907799/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-06-30_22:36:16-upgrade:pacific-x-wip-yuri3-testing-2022-06-28-1737-distro-default-smithi/6907799/ sentry: https://sentry.ceph.com/organizations/ceph/?query=36c3f4022af946d3997484a5f8e5b4c4 Command failed (workunit test cls/test_cls_rgw.sh) on smithi097 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=pacific TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh' Pass ================================================================= [6907792] upgrade:pacific-x/stress-split/{0-distro/rhel_8.6_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:11:02 info: http://pulpito.ceph.com/yuriw-2022-06-30_22:36:16-upgrade:pacific-x-wip-yuri3-testing-2022-06-28-1737-distro-default-smithi/6907792/ [6907794] upgrade:pacific-x/stress-split/{0-distro/centos_8.stream_container_tools 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:09:29 info: http://pulpito.ceph.com/yuriw-2022-06-30_22:36:16-upgrade:pacific-x-wip-yuri3-testing-2022-06-28-1737-distro-default-smithi/6907794/ [6907796] upgrade:pacific-x/stress-split/{0-distro/centos_8.stream_container_tools_crun 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:26:13 info: http://pulpito.ceph.com/yuriw-2022-06-30_22:36:16-upgrade:pacific-x-wip-yuri3-testing-2022-06-28-1737-distro-default-smithi/6907796/ [6907797] upgrade:pacific-x/stress-split/{0-distro/rhel_8.6_container_tools_3.0 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:08:29 info: http://pulpito.ceph.com/yuriw-2022-06-30_22:36:16-upgrade:pacific-x-wip-yuri3-testing-2022-06-28-1737-distro-default-smithi/6907797/ [6907798] upgrade:pacific-x/stress-split/{0-distro/rhel_8.6_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:53:13 info: http://pulpito.ceph.com/yuriw-2022-06-30_22:36:16-upgrade:pacific-x-wip-yuri3-testing-2022-06-28-1737-distro-default-smithi/6907798/ [6907800] upgrade:pacific-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:10:20 info: http://pulpito.ceph.com/yuriw-2022-06-30_22:36:16-upgrade:pacific-x-wip-yuri3-testing-2022-06-28-1737-distro-default-smithi/6907800/