2023-05-19T14:24:30.718 INFO:root:teuthology version: 0.0.1.dev98+g69caab8 2023-05-19T14:24:30.719 INFO:teuthology.results:subset: '111/120000' 2023-05-19T14:24:30.719 INFO:teuthology.results:seed: '6275' 2023-05-19T14:24:34.732 INFO:root:teuthology version: 0.0.1.dev98+g69caab8 2023-05-19T14:24:34.733 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-05-19T17:05:47.384 INFO:teuthology.results:Tests finished! gathering results... 2023-05-19T17:05:48.254 INFO:scrape:Found 9 jobs 2023-05-19T17:05:48.614 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/teuthology-2023-05-19_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/7278027/teuthology.log 2023-05-19T17:05:48.899 INFO:scrape:Found 2 distinct failure reasons 2023-05-19T17:05:48.900 INFO:scrape:Failure: Command failed (workunit test cls/test_cls_refcount.sh) on smithi046 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=pacific TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_refcount.sh' 2023-05-19T17:05:48.901 INFO:scrape:2 jobs: ['7278033', '7278029'] 2023-05-19T17:05:48.902 INFO:scrape:suites intersection: ['0-start', '1-tasks', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'test_rbd_api', 'test_rbd_python}}', 'upgrade-sequence', 'upgrade:pacific-x/parallel/{0-random-distro$/{centos_8.stream_container_tools_crun}', 'workload/{ec-rados-default'] 2023-05-19T17:05:48.902 INFO:scrape:suites union: ['0-start', '1-tasks', 'mon_election/classic', 'mon_election/connectivity', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'test_rbd_api', 'test_rbd_python}}', 'upgrade-sequence', 'upgrade:pacific-x/parallel/{0-random-distro$/{centos_8.stream_container_tools_crun}', 'workload/{ec-rados-default'] 2023-05-19T17:05:48.903 INFO:scrape: 2023-05-19T17:05:48.904 INFO:scrape:Dead: None 2023-05-19T17:05:48.905 INFO:scrape:1 jobs: ['7278027'] 2023-05-19T17:05:48.905 INFO:scrape: 2023-05-19T17:05:48.907 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2023-05-19_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2023-05-19_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2023-05-19_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/ failed: 2 dead: 1 running: 0 waiting: 0 queued: 0 passed: 6 Fail ================================================================= [7278029] upgrade:pacific-x/parallel/{0-random-distro$/{centos_8.stream_container_tools_crun} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} ----------------------------------------------------------------- time: 00:38:02 info: http://pulpito.ceph.com/teuthology-2023-05-19_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/7278029/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-05-19_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/7278029/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bcedfefe6dcd464bafab57700371b80e Command failed (workunit test cls/test_cls_refcount.sh) on smithi154 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=pacific TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_refcount.sh' [7278033] upgrade:pacific-x/parallel/{0-random-distro$/{centos_8.stream_container_tools_crun} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} ----------------------------------------------------------------- time: 00:36:37 info: http://pulpito.ceph.com/teuthology-2023-05-19_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/7278033/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-05-19_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/7278033/ sentry: https://sentry.ceph.com/organizations/ceph/?query=065bb08385024368bbf8d822f49d0180 Command failed (workunit test cls/test_cls_refcount.sh) on smithi046 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=pacific TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_refcount.sh' Dead ================================================================= [7278027] upgrade:pacific-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2023-05-19_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/7278027/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-05-19_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/7278027/ Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds Pass ================================================================= [7278026] upgrade:pacific-x/stress-split/{0-distro/rhel_8.4_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:14:36 info: http://pulpito.ceph.com/teuthology-2023-05-19_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/7278026/ [7278028] upgrade:pacific-x/stress-split/{0-distro/centos_8.stream_container_tools 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:05:44 info: http://pulpito.ceph.com/teuthology-2023-05-19_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/7278028/ [7278030] upgrade:pacific-x/stress-split/{0-distro/centos_8.stream_container_tools_crun 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:17:04 info: http://pulpito.ceph.com/teuthology-2023-05-19_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/7278030/ [7278031] upgrade:pacific-x/stress-split/{0-distro/rhel_8.4_container_tools_3.0 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:11:02 info: http://pulpito.ceph.com/teuthology-2023-05-19_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/7278031/ [7278032] upgrade:pacific-x/stress-split/{0-distro/rhel_8.4_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:17:00 info: http://pulpito.ceph.com/teuthology-2023-05-19_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/7278032/ [7278034] upgrade:pacific-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:21:03 info: http://pulpito.ceph.com/teuthology-2023-05-19_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/7278034/