Found 307 jobs
Misunderstood line: 
Misunderstood line: 
Missing teuthology log /home/teuthworker/archive/yuriw-2023-07-24_21:23:58-rados-wip-yuri5-testing-2023-07-24-0814-distro-default-smithi/7350132/teuthology.log
Misunderstood line: 
Found 16 distinct failure reasons
Failure: reached maximum tries (51) after waiting for 300 seconds
1 jobs: ['7350068']
suites: ['2-inject-leak/none', 'centos_latest}', 'rados/valgrind-leaks/{1-start']

Valgrind: mon (Leak_StillReachable)
3 jobs: ['7350003', '7350138', '7349931']
suites intersection: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'validater/valgrind}']
suites union: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async', 'msgr/async-v1only', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_api_tests', 'tasks/rados_cls_all', 'validater/valgrind}']

Dead: None
17 jobs: ['7350132', '7350065', '7350161', '7350017', '7350109', '7350163', '7350111', '7350113', '7350091', '7350069', '7350114', '7350049', '7350018', '7350019', '7350131', '7350115', '7349985']

Dead: b'2023-07-25T11:57:01.516 DEBUG:teuthology.exit:Finished running handlers'
3 jobs: ['7350121', '7349967', '7350093']
suites intersection: ['rados']
suites union: ['clusters/{fixed-2', 'fast/fast', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/osd-delay', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/singleton-nomsgr/{all/admin_socket_output', 'rados/thrash-erasure-code/{ceph', 'recovery-overrides/{default}', 'supported-random-distro$/{centos_latest}}', 'supported-random-distro$/{ubuntu_20.04}}', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/default', 'thrashosds-health', 'workloads/ec-small-objects-balanced}']

Failure: Stale jobs detected, aborting.
2 jobs: ['7350050', '7350112']
suites intersection: ['mon_election/connectivity', 'rados']
suites union: ['ceph', 'clusters/{fixed-2', 'fast/normal', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/many', 'msgr/async', 'no_pools', 'objectstore/bluestore-comp-snappy', 'openstack}', 'rados', 'rados/multimon/{clusters/3', 'rados/thrash-erasure-code-overwrites/{bluestore-bitmap', 'recovery-overrides/{more-async-recovery}', 'supported-random-distro$/{ubuntu_20.04}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/mon_clock_no_skews}', 'thrashers/default', 'thrashosds-health', 'workloads/ec-small-objects-fast-read-overwrites}']

Failure: {'useradd: warning: the home directory already exists.', 'Not copying any file from skel directory into it.', "useradd: user 'svelar' already exists", "useradd: user 'vedansh' already exists"}
2 jobs: ['7350016', '7350015']
suites intersection: ['mon_election/classic', 'rados']
suites union: ['cluster/{12-osds', 'mon_election/classic', 'msgr-failures/few', 'objectstore/bluestore-comp-lz4', 'openstack}', 'rados', 'rados/singleton-nomsgr/{all/export-after-evict', 'rados/thrash-erasure-code-big/{ceph', 'recovery-overrides/{more-async-recovery}', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{rhel_8}}', 'thrashers/morepggrow', 'thrashosds-health', 'workloads/ec-rados-plugin=clay-k=4-m=2}']

Failure: Test failure: test_cluster_info (tasks.cephfs.test_nfs.TestNFS)
2 jobs: ['7349944', '7350088']
suites intersection: ['agent/on', 'mon_election/classic', 'task/test_nfs}']
suites union: ['agent/on', 'mon_election/classic', 'rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8', 'rados/cephadm/workunits/{0-distro/ubuntu_20.04', 'task/test_nfs}']

Failure: {'/dev/vg_nvme: already exists in filesystem', "Creating volume group 'vg_nvme' failed", "Run `vgcreate --help' for more information."}
1 jobs: ['7350020']
suites: ['mon_election/classic', 'objectstore/bluestore-bitmap', 'openstack', 'rados/perf/{ceph', 'scheduler/dmclock_default_shards', 'settings/optimized', 'ubuntu_latest', 'workloads/radosbench_4M_rand_read}']

Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi039 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=fdae5f8121ac5992c997226a81d2e06f029ab247 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh'
2 jobs: ['7350198', '7350042']
suites intersection: ['debug/mgr', 'rados/dashboard/{0-single-container-host', 'tasks/e2e}']
suites union: ['debug/mgr', 'mon_election/classic', 'mon_election/connectivity', 'rados/dashboard/{0-single-container-host', 'random-objectstore$/{bluestore-bitmap}', 'random-objectstore$/{bluestore-comp-lz4}', 'tasks/e2e}']

Failure: Command failed on smithi055 with status 1: 'sudo fuser -v /var/lib/dpkg/lock-frontend'
1 jobs: ['7350165']
suites: ['clusters/{fixed-2', 'mon_election/classic', 'msgr-failures/many', 'msgr/async-v2only', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/basic/{ceph', 'supported-random-distro$/{ubuntu_latest}', 'tasks/repair_test}']

Failure: [Errno None] Unable to connect to port 22 on 172.21.15.193
1 jobs: ['7350092']
suites: ['mon_election/connectivity', 'rados', 'rados/singleton-nomsgr/{all/pool-access', 'supported-random-distro$/{ubuntu_20.04}}']

Dead: b'OSError: Socket is closed'
1 jobs: ['7350196']
suites: ['ceph', 'clusters/{fixed-2', 'mon_election/connectivity', 'msgr-failures/few', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/thrash-erasure-code-isa/{arch/x86_64', 'recovery-overrides/{more-async-recovery}', 'supported-random-distro$/{centos_latest}', 'thrashers/mapgap', 'thrashosds-health', 'workloads/ec-rados-plugin=isa-k=2-m=1}']

Failure: SELinux denials found on ubuntu@smithi062.front.sepia.ceph.com: ['type=AVC msg=audit(1690246581.829:20761): avc:  denied  { ioctl } for  pid=112609 comm="iptables" path="/var/lib/containers/storage/overlay/5382d711f9695261e1f0b72651a3f239cfab350dc2a4be3d2be6ac93be13578c/merged" dev="overlay" ino=2753998 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1']
1 jobs: ['7350173']
suites: ['agent/on', 'mon_election/classic', 'rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8', 'task/test_cephadm}']

Timeout 3h running clone.client.0/qa/workunits/cls/test_cls_2pc_queue.sh
2 jobs: ['7349969', '7350123']
suites intersection: ['0-start', '1-tasks', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'test_rbd_api', 'test_rbd_python}}', 'upgrade-sequence', 'workload/{ec-rados-default']
suites union: ['0-start', '1-tasks', 'mon_election/classic', 'mon_election/connectivity', 'rados/upgrade/parallel/{0-random-distro$/{centos_8.stream_container_tools_crun}', 'rados/upgrade/parallel/{0-random-distro$/{centos_8.stream_container_tools}', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'test_rbd_api', 'test_rbd_python}}', 'upgrade-sequence', 'workload/{ec-rados-default']

Failure: SSH connection to smithi092 was lost: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-5090-gfdae5f81-1focal cephadm=18.0.0-5090-gfdae5f81-1focal ceph-mds=18.0.0-5090-gfdae5f81-1focal ceph-mgr=18.0.0-5090-gfdae5f81-1focal ceph-common=18.0.0-5090-gfdae5f81-1focal ceph-fuse=18.0.0-5090-gfdae5f81-1focal ceph-test=18.0.0-5090-gfdae5f81-1focal ceph-volume=18.0.0-5090-gfdae5f81-1focal radosgw=18.0.0-5090-gfdae5f81-1focal python3-rados=18.0.0-5090-gfdae5f81-1focal python3-rgw=18.0.0-5090-gfdae5f81-1focal python3-cephfs=18.0.0-5090-gfdae5f81-1focal python3-rbd=18.0.0-5090-gfdae5f81-1focal libcephfs2=18.0.0-5090-gfdae5f81-1focal libcephfs-dev=18.0.0-5090-gfdae5f81-1focal librados2=18.0.0-5090-gfdae5f81-1focal librbd1=18.0.0-5090-gfdae5f81-1focal rbd-fuse=18.0.0-5090-gfdae5f81-1focal'
1 jobs: ['7350122']
suites: ['rados/standalone/{supported-random-distro$/{ubuntu_20.04}', 'workloads/mon-stretch}']

Failure: 
1 jobs: ['7350141']
suites: ['0-nvme-loop', 'agent/on', 'fixed-2', 'mon_election/connectivity', 'rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0', 'start}']