2024-04-09T11:53:45.960 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-04-09T11:53:45.960 INFO:teuthology.results:subset: '111/120000' 2024-04-09T11:53:45.960 INFO:teuthology.results:seed: '4901' 2024-04-09T11:58:42.477 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-04-09T11:58:42.480 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2024-04-09T17:00:06.201 INFO:teuthology.results:Tests finished! gathering results... 2024-04-09T17:00:06.663 INFO:scrape:Found 5 jobs 2024-04-09T17:00:07.046 INFO:scrape:Found 2 distinct failure reasons 2024-04-09T17:00:07.046 INFO:scrape:Failure: No module named 'tasks' 2024-04-09T17:00:07.046 INFO:scrape:3 jobs: ['7647505', '7647507', '7647509'] 2024-04-09T17:00:07.046 INFO:scrape:suites intersection: [] 2024-04-09T17:00:07.047 INFO:scrape:suites union: ['1-ceph-install/reef', '1.1.short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{fsx', '5-finish-upgrade', '6-final-workload/{rbd-python', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp', 'radosbench', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'supported-all-distro/ubuntu_latest', 'supported-all-distro/ubuntu_latest}', 'thrashosds-health}', 'upgrade:reef-p2p/reef-p2p-parallel/{point-to-point-upgrade', 'upgrade:reef-p2p/reef-p2p-stress-split/{0-cluster/{openstack'] 2024-04-09T17:00:07.047 INFO:scrape: 2024-04-09T17:00:07.047 INFO:scrape:Timeout 3h running clone.client.1/qa/workunits/cls/test_cls_2pc_queue.sh 2024-04-09T17:00:07.047 INFO:scrape:1 jobs: ['7647503'] 2024-04-09T17:00:07.047 INFO:scrape:suites: ['supported-all-distro/centos_8}', 'upgrade:reef-p2p/reef-p2p-parallel/{point-to-point-upgrade'] 2024-04-09T17:00:07.047 INFO:scrape: 2024-04-09T17:00:07.048 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2024-04-09_01:18:39-upgrade:reef-p2p-reef-release-distro-default-smithi ================================================================= info: https://pulpito.ceph.com/yuriw-2024-04-09_01:18:39-upgrade:reef-p2p-reef-release-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2024-04-09_01:18:39-upgrade:reef-p2p-reef-release-distro-default-smithi/ failed: 4 dead: 0 running: 0 waiting: 0 queued: 0 passed: 1 Fail ================================================================= [7647503] upgrade:reef-p2p/reef-p2p-parallel/{point-to-point-upgrade supported-all-distro/centos_8} ----------------------------------------------------------------- time: 04:55:43 info: https://pulpito.ceph.com/yuriw-2024-04-09_01:18:39-upgrade:reef-p2p-reef-release-distro-default-smithi/7647503/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-04-09_01:18:39-upgrade:reef-p2p-reef-release-distro-default-smithi/7647503/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8e4623d0589b42138d9be136d11d6755 Command failed (workunit test cls/test_cls_2pc_queue.sh) on smithi194 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=reef TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 CLS_RBD_GTEST_FILTER=\'*:-TestClsRbd.mirror_snapshot\' adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cep htest/clone.client.1/qa/workunits/cls/test_cls_2pc_queue.sh' [7647505] upgrade:reef-p2p/reef-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/reef 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 6-final-workload/{rbd-python snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/ubuntu_latest thrashosds-health} ----------------------------------------------------------------- time: 00:07:38 info: https://pulpito.ceph.com/yuriw-2024-04-09_01:18:39-upgrade:reef-p2p-reef-release-distro-default-smithi/7647505/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-04-09_01:18:39-upgrade:reef-p2p-reef-release-distro-default-smithi/7647505/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ac59ec1980064451b6014f58f5d68173 No module named 'tasks' [7647507] upgrade:reef-p2p/reef-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/reef 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 6-final-workload/{rbd-python snaps-many-objects} objectstore/bluestore-comp supported-all-distro/ubuntu_latest thrashosds-health} ----------------------------------------------------------------- time: 00:07:54 info: https://pulpito.ceph.com/yuriw-2024-04-09_01:18:39-upgrade:reef-p2p-reef-release-distro-default-smithi/7647507/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-04-09_01:18:39-upgrade:reef-p2p-reef-release-distro-default-smithi/7647507/ sentry: https://sentry.ceph.com/organizations/ceph/?query=961e3d026f80422d94747c1cb190303d No module named 'tasks' [7647509] upgrade:reef-p2p/reef-p2p-parallel/{point-to-point-upgrade supported-all-distro/ubuntu_latest} ----------------------------------------------------------------- time: 00:07:40 info: https://pulpito.ceph.com/yuriw-2024-04-09_01:18:39-upgrade:reef-p2p-reef-release-distro-default-smithi/7647509/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-04-09_01:18:39-upgrade:reef-p2p-reef-release-distro-default-smithi/7647509/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f90c472d0dc945f1812c60a58340d2da No module named 'tasks' Pass ================================================================= [7647511] upgrade:reef-p2p/reef-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/reef 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 6-final-workload/{rbd-python snaps-many-objects} objectstore/bluestore-stupid supported-all-distro/ubuntu_latest thrashosds-health} time: 03:07:53 info: https://pulpito.ceph.com/yuriw-2024-04-09_01:18:39-upgrade:reef-p2p-reef-release-distro-default-smithi/7647511/