Found 27 jobs
Missing teuthology log /home/teuthworker/archive/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294325/teuthology.log
Found 7 distinct failure reasons
Failure: Command failed (workunit test cls/test_cls_rbd.sh) on smithi105 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=octopus TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh'
15 jobs: ['7294327', '7294306', '7294310', '7294317', '7294319', '7294308', '7294313', '7294323', '7294311', '7294309', '7294318', '7294322', '7294331', '7294312', '7294330']
suites intersection: []
suites union: ['0-roles', '0-start', '1-ceph-install/octopus', '1-start', '1-tasks', '1.1-pg-log-overrides/normal_pg_log', '1.1-pg-log-overrides/short_pg_log', '2-first-half-tasks/rbd-import-export', '2-first-half-tasks/rbd_api', '2-first-half-tasks/readwrite', '2-partial-upgrade/firsthalf', '3-stress-tasks/{radosbench', '3-thrash/default', '4-second-half-tasks/radosbench', '4-second-half-tasks/rbd-import-export', '4-workload/{radosbench', '5-finish-upgrade', '6-quincy', '8-final-workload/{rbd-python', 'mon_election/classic', 'mon_election/classic}', 'mon_election/connectivity', 'mon_election/connectivity}', 'objectstore/bluestore-hybrid', 'objectstore/filestore-xfs', 'rados_api', 'rados_loadgenbig', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'rbd_import_export', 'readwrite', 'rgw_ragweed_prepare', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'test_rbd_api}}', 'thrashosds-health', 'ubuntu_20.04}', 'upgrade-sequence', 'upgrade:octopus-x/parallel/{0-distro/centos_8.stream_container_tools', 'upgrade:octopus-x/parallel/{0-distro/centos_8.stream_container_tools_crun', 'upgrade:octopus-x/parallel/{0-distro/rhel_8.4_container_tools_3.0', 'upgrade:octopus-x/parallel/{0-distro/rhel_8.4_container_tools_rhel8', 'upgrade:octopus-x/parallel/{0-distro/ubuntu_20.04', 'upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack', 'upgrade:octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_3.0', 'upgrade:octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_rhel8', 'upgrade:octopus-x/stress-split/{0-distro/ubuntu_20.04', 'workload/{ec-rados-default']

Dead: b"2023-06-02T14:51:59.908 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi023.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294315', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-02 14:23:47.753279', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}"
1 jobs: ['7294315']
suites: ['1-octopus-install/octopus', '1.1-pg-log-overrides/short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '3.1-objectstore/bluestore-hybrid', '4-ec-workload/{rados-ec-workload', '5-finish-upgrade', '6-quincy', '7-final-workload', 'mon_election/classic', 'rbd-ec-workload}', 'start}', 'thrashosds-health', 'ubuntu_20.04}', 'upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack']

Failure: Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=octopus
2 jobs: ['7294314', '7294328']
suites intersection: ['frontend', 'overrides', 'realm', 'tasks', 'upgrade:octopus-x/rgw-multisite/{clusters']
suites union: ['frontend', 'overrides', 'realm', 'tasks', 'upgrade/primary}', 'upgrade/secondary}', 'upgrade:octopus-x/rgw-multisite/{clusters']

Dead: None
1 jobs: ['7294325']

Failure: {'smithi067.front.sepia.ceph.com': {'changed': False, 'msg': 'All items completed', 'results': [{'_ansible_item_label': ['cpanminus', 'perl-doc'], '_ansible_no_log': False, 'ansible_loop_var': 'item', 'cache_update_time': 1685716654, 'cache_updated': False, 'changed': False, 'failed': True, 'invocation': {'module_args': {'allow_unauthenticated': False, 'autoclean': False, 'autoremove': False, 'cache_valid_time': 0, 'deb': None, 'default_release': None, 'dpkg_options': 'force-confdef,force-confold', 'force': False, 'force_apt_get': False, 'install_recommends': None, 'name': ['cpanminus', 'perl-doc'], 'only_upgrade': False, 'package': ['cpanminus', 'perl-doc'], 'policy_rc_d': None, 'purge': False, 'state': 'present', 'update_cache': None, 'update_cache_retries': 5, 'update_cache_retry_max_delay': 12, 'upgrade': None}}, 'item': ['cpanminus', 'perl-doc'], 'msg': '\'/usr/bin/apt-get -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold"      install \'cpanminus\'\' failed: E: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 20571 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n', 'rc': 100, 'stderr': 'E: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 20571 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n', 'stderr_lines': ['E: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 20571 (apt-get)', 'E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?'], 'stdout': '', 'stdout_lines': []}]}}
1 jobs: ['7294316']
suites: ['1-ceph-install/octopus', '1.1-pg-log-overrides/short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{radosbench', '5-finish-upgrade', '6-quincy', '8-final-workload/{rbd-python', 'mon_election/classic', 'objectstore/bluestore-hybrid', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw_ragweed_prepare', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'thrashosds-health', 'ubuntu_20.04}', 'upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack']

Failure: Command failed on smithi097 with status 95: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:octopus shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f01e60-0155-11ee-9b1d-001a4aab830c -e sha1=850f162d779001975d92ecad3d1ae9dffcdd1eef -- bash -c 'ceph orch ps'"
2 jobs: ['7294320', '7294324']
suites intersection: ['0-roles', '1-start', '3-stress-tasks/{radosbench', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}']
suites union: ['0-roles', '1-start', '2-first-half-tasks/radosbench', '2-first-half-tasks/snaps-few-objects', '3-stress-tasks/{radosbench', '4-second-half-tasks/radosbench', '4-second-half-tasks/rbd-import-export', 'mon_election/classic}', 'mon_election/connectivity}', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}', 'upgrade:octopus-x/stress-split/{0-distro/centos_8.stream_container_tools', 'upgrade:octopus-x/stress-split/{0-distro/centos_8.stream_container_tools_crun']

Failure: SSH connection to smithi169 was lost: 'sudo yum install -y kernel'
1 jobs: ['7294326']
suites: ['0-roles', '1-start', '2-first-half-tasks/rbd-cls', '3-stress-tasks/{radosbench', '4-second-half-tasks/radosbench', 'mon_election/connectivity}', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}', 'upgrade:octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_3.0']