2023-06-02T14:23:00.283 INFO:root:teuthology version: 0.0.1.dev116+g62814ab 2023-06-02T14:23:00.284 INFO:teuthology.results:subset: '111/120000' 2023-06-02T14:23:00.284 INFO:teuthology.results:seed: '2294' 2023-06-02T14:23:57.466 INFO:root:teuthology version: 0.0.1.dev116+g62814ab 2023-06-02T14:23:57.470 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-06-02T18:01:14.606 INFO:teuthology.results:Tests finished! gathering results... 2023-06-02T18:01:17.618 INFO:scrape:Found 27 jobs 2023-06-02T18:01:20.307 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294325/teuthology.log 2023-06-02T18:01:25.262 INFO:scrape:Found 7 distinct failure reasons 2023-06-02T18:01:25.265 INFO:scrape:Failure: Command failed (workunit test cls/test_cls_rbd.sh) on smithi105 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=octopus TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh' 2023-06-02T18:01:25.266 INFO:scrape:15 jobs: ['7294327', '7294306', '7294310', '7294317', '7294319', '7294308', '7294313', '7294323', '7294311', '7294309', '7294318', '7294322', '7294331', '7294312', '7294330'] 2023-06-02T18:01:25.266 INFO:scrape:suites intersection: [] 2023-06-02T18:01:25.272 INFO:scrape:suites union: ['0-roles', '0-start', '1-ceph-install/octopus', '1-start', '1-tasks', '1.1-pg-log-overrides/normal_pg_log', '1.1-pg-log-overrides/short_pg_log', '2-first-half-tasks/rbd-import-export', '2-first-half-tasks/rbd_api', '2-first-half-tasks/readwrite', '2-partial-upgrade/firsthalf', '3-stress-tasks/{radosbench', '3-thrash/default', '4-second-half-tasks/radosbench', '4-second-half-tasks/rbd-import-export', '4-workload/{radosbench', '5-finish-upgrade', '6-quincy', '8-final-workload/{rbd-python', 'mon_election/classic', 'mon_election/classic}', 'mon_election/connectivity', 'mon_election/connectivity}', 'objectstore/bluestore-hybrid', 'objectstore/filestore-xfs', 'rados_api', 'rados_loadgenbig', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'rbd_import_export', 'readwrite', 'rgw_ragweed_prepare', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'test_rbd_api}}', 'thrashosds-health', 'ubuntu_20.04}', 'upgrade-sequence', 'upgrade:octopus-x/parallel/{0-distro/centos_8.stream_container_tools', 'upgrade:octopus-x/parallel/{0-distro/centos_8.stream_container_tools_crun', 'upgrade:octopus-x/parallel/{0-distro/rhel_8.4_container_tools_3.0', 'upgrade:octopus-x/parallel/{0-distro/rhel_8.4_container_tools_rhel8', 'upgrade:octopus-x/parallel/{0-distro/ubuntu_20.04', 'upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack', 'upgrade:octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_3.0', 'upgrade:octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_rhel8', 'upgrade:octopus-x/stress-split/{0-distro/ubuntu_20.04', 'workload/{ec-rados-default'] 2023-06-02T18:01:25.273 INFO:scrape: 2023-06-02T18:01:25.273 INFO:scrape:Dead: b"2023-06-02T14:51:59.908 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi023.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294315', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-02 14:23:47.753279', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}" 2023-06-02T18:01:25.273 INFO:scrape:1 jobs: ['7294315'] 2023-06-02T18:01:25.274 INFO:scrape:suites: ['1-octopus-install/octopus', '1.1-pg-log-overrides/short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '3.1-objectstore/bluestore-hybrid', '4-ec-workload/{rados-ec-workload', '5-finish-upgrade', '6-quincy', '7-final-workload', 'mon_election/classic', 'rbd-ec-workload}', 'start}', 'thrashosds-health', 'ubuntu_20.04}', 'upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack'] 2023-06-02T18:01:25.274 INFO:scrape: 2023-06-02T18:01:25.274 INFO:scrape:Failure: Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=octopus 2023-06-02T18:01:25.275 INFO:scrape:2 jobs: ['7294314', '7294328'] 2023-06-02T18:01:25.275 INFO:scrape:suites intersection: ['frontend', 'overrides', 'realm', 'tasks', 'upgrade:octopus-x/rgw-multisite/{clusters'] 2023-06-02T18:01:25.275 INFO:scrape:suites union: ['frontend', 'overrides', 'realm', 'tasks', 'upgrade/primary}', 'upgrade/secondary}', 'upgrade:octopus-x/rgw-multisite/{clusters'] 2023-06-02T18:01:25.275 INFO:scrape: 2023-06-02T18:01:25.276 INFO:scrape:Dead: None 2023-06-02T18:01:25.276 INFO:scrape:1 jobs: ['7294325'] 2023-06-02T18:01:25.276 INFO:scrape: 2023-06-02T18:01:25.277 INFO:scrape:Failure: {'smithi067.front.sepia.ceph.com': {'changed': False, 'msg': 'All items completed', 'results': [{'_ansible_item_label': ['cpanminus', 'perl-doc'], '_ansible_no_log': False, 'ansible_loop_var': 'item', 'cache_update_time': 1685716654, 'cache_updated': False, 'changed': False, 'failed': True, 'invocation': {'module_args': {'allow_unauthenticated': False, 'autoclean': False, 'autoremove': False, 'cache_valid_time': 0, 'deb': None, 'default_release': None, 'dpkg_options': 'force-confdef,force-confold', 'force': False, 'force_apt_get': False, 'install_recommends': None, 'name': ['cpanminus', 'perl-doc'], 'only_upgrade': False, 'package': ['cpanminus', 'perl-doc'], 'policy_rc_d': None, 'purge': False, 'state': 'present', 'update_cache': None, 'update_cache_retries': 5, 'update_cache_retry_max_delay': 12, 'upgrade': None}}, 'item': ['cpanminus', 'perl-doc'], 'msg': '\'/usr/bin/apt-get -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" install \'cpanminus\'\' failed: E: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 20571 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n', 'rc': 100, 'stderr': 'E: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 20571 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n', 'stderr_lines': ['E: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 20571 (apt-get)', 'E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?'], 'stdout': '', 'stdout_lines': []}]}} 2023-06-02T18:01:25.277 INFO:scrape:1 jobs: ['7294316'] 2023-06-02T18:01:25.277 INFO:scrape:suites: ['1-ceph-install/octopus', '1.1-pg-log-overrides/short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{radosbench', '5-finish-upgrade', '6-quincy', '8-final-workload/{rbd-python', 'mon_election/classic', 'objectstore/bluestore-hybrid', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw_ragweed_prepare', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'thrashosds-health', 'ubuntu_20.04}', 'upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack'] 2023-06-02T18:01:25.278 INFO:scrape: 2023-06-02T18:01:25.278 INFO:scrape:Failure: Command failed on smithi097 with status 95: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:octopus shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f01e60-0155-11ee-9b1d-001a4aab830c -e sha1=850f162d779001975d92ecad3d1ae9dffcdd1eef -- bash -c 'ceph orch ps'" 2023-06-02T18:01:25.278 INFO:scrape:2 jobs: ['7294320', '7294324'] 2023-06-02T18:01:25.278 INFO:scrape:suites intersection: ['0-roles', '1-start', '3-stress-tasks/{radosbench', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}'] 2023-06-02T18:01:25.279 INFO:scrape:suites union: ['0-roles', '1-start', '2-first-half-tasks/radosbench', '2-first-half-tasks/snaps-few-objects', '3-stress-tasks/{radosbench', '4-second-half-tasks/radosbench', '4-second-half-tasks/rbd-import-export', 'mon_election/classic}', 'mon_election/connectivity}', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}', 'upgrade:octopus-x/stress-split/{0-distro/centos_8.stream_container_tools', 'upgrade:octopus-x/stress-split/{0-distro/centos_8.stream_container_tools_crun'] 2023-06-02T18:01:25.279 INFO:scrape: 2023-06-02T18:01:25.280 INFO:scrape:Failure: SSH connection to smithi169 was lost: 'sudo yum install -y kernel' 2023-06-02T18:01:25.280 INFO:scrape:1 jobs: ['7294326'] 2023-06-02T18:01:25.281 INFO:scrape:suites: ['0-roles', '1-start', '2-first-half-tasks/rbd-cls', '3-stress-tasks/{radosbench', '4-second-half-tasks/radosbench', 'mon_election/connectivity}', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}', 'upgrade:octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_3.0'] 2023-06-02T18:01:25.281 INFO:scrape: 2023-06-02T18:01:25.284 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/ failed: 19 dead: 4 running: 0 waiting: 0 queued: 0 passed: 4 Fail ================================================================= [7294306] upgrade:octopus-x/parallel/{0-distro/rhel_8.4_container_tools_3.0 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} ----------------------------------------------------------------- time: 00:42:57 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294306/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294306/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f9b6f0cfb14f4b8a8e88d609f5eac965 Command failed (workunit test cls/test_cls_rbd.sh) on smithi119 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=octopus TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh' [7294308] upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-quincy 8-final-workload/{rbd-python snaps-many-objects} mon_election/classic objectstore/filestore-xfs thrashosds-health ubuntu_20.04} ----------------------------------------------------------------- time: 01:34:24 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294308/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294308/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b4bbc2e377e446159d9cad395285c305 Command failed (workunit test cls/test_cls_rbd.sh) on smithi192 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=octopus TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh' [7294309] upgrade:octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_3.0 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} ----------------------------------------------------------------- time: 01:41:29 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294309/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294309/ sentry: https://sentry.ceph.com/organizations/ceph/?query=905ab2cdb738463d81120e6a7f8de7a1 Command failed (workunit test cls/test_cls_rbd.sh) on smithi189 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=octopus TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh' [7294310] upgrade:octopus-x/parallel/{0-distro/rhel_8.4_container_tools_rhel8 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} ----------------------------------------------------------------- time: 00:39:34 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294310/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294310/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fbad70ea533a49ce99babea120ca0d1f Command failed (workunit test cls/test_cls_rbd.sh) on smithi026 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=octopus TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh' [7294311] upgrade:octopus-x/parallel/{0-distro/ubuntu_20.04 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} ----------------------------------------------------------------- time: 00:36:36 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294311/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294311/ sentry: https://sentry.ceph.com/organizations/ceph/?query=59f23708af974138a877cbbbbcddab60 Command failed (workunit test cls/test_cls_rbd.sh) on smithi038 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=octopus TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh' [7294312] upgrade:octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} ----------------------------------------------------------------- time: 01:40:02 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294312/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294312/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a9ed0eddd7bb4e0396d4545f7cdcd797 Command failed (workunit test cls/test_cls_rbd.sh) on smithi071 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=octopus TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh' [7294313] upgrade:octopus-x/parallel/{0-distro/centos_8.stream_container_tools 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} ----------------------------------------------------------------- time: 00:41:21 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294313/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294313/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4a8cd6b2e95643f4b29a3bab7fd02d90 Command failed (workunit test cls/test_cls_rbd.sh) on smithi090 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=octopus TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh' [7294314] upgrade:octopus-x/rgw-multisite/{clusters frontend overrides realm tasks upgrade/primary} ----------------------------------------------------------------- time: 00:07:45 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294314/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294314/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ee0829139f8b4679b0aa4c506bb7b211 Failed to fetch package version from https://shaman.ceph.com/api/search/?st atus=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref= octopus [7294317] upgrade:octopus-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} ----------------------------------------------------------------- time: 01:36:15 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294317/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294317/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2aaaba08f2e449dcbefac3c95d6efe03 Command failed (workunit test cls/test_cls_rbd.sh) on smithi089 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=octopus TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh' [7294318] upgrade:octopus-x/parallel/{0-distro/centos_8.stream_container_tools_crun 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} ----------------------------------------------------------------- time: 00:38:42 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294318/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294318/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9218bc81882f45808f4192cb477ce2bb Command failed (workunit test cls/test_cls_rbd.sh) on smithi060 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=octopus TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh' [7294319] upgrade:octopus-x/parallel/{0-distro/rhel_8.4_container_tools_3.0 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} ----------------------------------------------------------------- time: 00:44:30 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294319/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294319/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8f0328f4986449da8d32c52f128b94d8 Command failed (workunit test cls/test_cls_rbd.sh) on smithi081 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=octopus TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh' [7294320] upgrade:octopus-x/stress-split/{0-distro/centos_8.stream_container_tools 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} ----------------------------------------------------------------- time: 00:55:41 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294320/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294320/ sentry: https://sentry.ceph.com/organizations/ceph/?query=088dcb46644e4a4988de8fae5c024d5a Command failed on smithi097 with status 95: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:octopus shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f01e60-0155-11ee-9b1d-001a4aab830c -e sha1=850f162d779001975d92ecad3d1ae9dffcdd1eef -- bash -c 'ceph orch ps'" [7294322] upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-quincy 8-final-workload/{rbd-python snaps-many-objects} mon_election/connectivity objectstore/filestore-xfs thrashosds-health ubuntu_20.04} ----------------------------------------------------------------- time: 01:38:52 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294322/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294322/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e00dcea47a8a47deb5ed32c53a0d5baf Command failed (workunit test cls/test_cls_rbd.sh) on smithi062 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=octopus TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh' [7294323] upgrade:octopus-x/parallel/{0-distro/rhel_8.4_container_tools_rhel8 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} ----------------------------------------------------------------- time: 00:37:19 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294323/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294323/ sentry: https://sentry.ceph.com/organizations/ceph/?query=be5f32e22bd84d859d6cf7759f98defc Command failed (workunit test cls/test_cls_rbd.sh) on smithi153 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=octopus TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh' [7294324] upgrade:octopus-x/stress-split/{0-distro/centos_8.stream_container_tools_crun 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} ----------------------------------------------------------------- time: 00:59:45 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294324/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294324/ sentry: https://sentry.ceph.com/organizations/ceph/?query=517000b8e536439d87577f769468097f Command failed on smithi022 with status 95: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:octopus shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 71756856-0155-11ee-9b1d-001a4aab830c -e sha1=850f162d779001975d92ecad3d1ae9dffcdd1eef -- bash -c 'ceph orch ps'" [7294327] upgrade:octopus-x/parallel/{0-distro/centos_8.stream_container_tools 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} ----------------------------------------------------------------- time: 00:34:05 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294327/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294327/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f72065a1bb7148448a1eb78dd2094c21 Command failed (workunit test cls/test_cls_rbd.sh) on smithi105 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=octopus TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh' [7294328] upgrade:octopus-x/rgw-multisite/{clusters frontend overrides realm tasks upgrade/secondary} ----------------------------------------------------------------- time: 00:08:23 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294328/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294328/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1b6da7fa8c21461fa14bc43b6e520918 Failed to fetch package version from https://shaman.ceph.com/api/search/?st atus=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref= octopus [7294330] upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-quincy 8-final-workload/{rbd-python snaps-many-objects} mon_election/connectivity objectstore/bluestore-hybrid thrashosds-health ubuntu_20.04} ----------------------------------------------------------------- time: 03:16:01 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294330/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294330/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b64dc51000e34744b4f3b6e327b9e0b9 Command failed (workunit test cls/test_cls_rbd.sh) on smithi163 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=octopus TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh' [7294331] upgrade:octopus-x/parallel/{0-distro/centos_8.stream_container_tools_crun 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} ----------------------------------------------------------------- time: 00:34:30 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294331/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294331/ sentry: https://sentry.ceph.com/organizations/ceph/?query=88fca66b6e0340e19d6455b50e544bbf Command failed (workunit test cls/test_cls_rbd.sh) on smithi098 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=octopus TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh' Dead ================================================================= [7294315] upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-hybrid 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-quincy 7-final-workload mon_election/classic thrashosds-health ubuntu_20.04} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294315/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294315/ [7294316] upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-quincy 8-final-workload/{rbd-python snaps-many-objects} mon_election/classic objectstore/bluestore-hybrid thrashosds-health ubuntu_20.04} ----------------------------------------------------------------- time: 00:06:22 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294316/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294316/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6328d1f73752469c8847a35d69ea560c {'smithi067.front.sepia.ceph.com': {'changed': False, 'msg': 'All items completed', 'results': [{'_ansible_item_label': ['cpanminus', 'perl-doc'], '_ansible_no_log': False, 'ansible_loop_var': 'item', 'cache_update_time': 1685716654, 'cache_updated': False, 'changed': False, 'failed': True, 'invocation': {'module_args': {'allow_unauthenticated': False, 'autoclean': False, 'autoremove': False, 'cache_valid_time': 0, 'deb': None, 'default_release': None, 'dpkg_options': 'force-confdef,force-confold', 'force': False, 'force_apt_get': False, 'install_recommends': None, 'name': ['cpanminus', 'perl-doc'], 'only_upgrade': False, 'package': ['cpanminus', 'perl-doc'], 'policy_rc_d': None, 'purge': False, 'state': 'present', 'update_cache': None, 'update_cache_retries': 5, 'update_cache_retry_max_delay': 12, 'upgrade': None}}, 'item': ['cpanminus', 'perl-doc'], 'msg': '\'/usr/bin/apt-get -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" install \'cpanminus\'\' failed: E: Could not get lock /var/lib/dpkg/lock- frontend. It is held by process 20571 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n', 'rc': 100, 'stderr': 'E: Could not get lock /var/lib/dpkg/lock- frontend. It is held by process 20571 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n', 'stderr_lines': ['E: Could not get lock /var/lib/dpkg/lock- frontend. It is held by process 20571 (apt-get)', 'E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?'], 'stdout': '', 'stdout_lines': []}]}} [7294325] upgrade:octopus-x/parallel/{0-distro/ubuntu_20.04 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294325/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294325/ Error reimaging machines: Expected smithi169's OS to be ubuntu 20.04 but found rhel 8.4 [7294326] upgrade:octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_3.0 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294326/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294326/ sentry: https://sentry.ceph.com/organizations/ceph/?query=186c32b01e8e4f09a9b9feeee713d630 SSH connection to smithi169 was lost: 'sudo yum install -y kernel' Pass ================================================================= [7294307] upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-hybrid 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-quincy 7-final-workload mon_election/connectivity thrashosds-health ubuntu_20.04} time: 02:39:05 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294307/ [7294321] upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-quincy 7-final-workload mon_election/connectivity thrashosds-health ubuntu_20.04} time: 02:18:32 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294321/ [7294329] upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-quincy 7-final-workload mon_election/classic thrashosds-health ubuntu_20.04} time: 01:42:41 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294329/ [7294332] upgrade:octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:03:24 info: http://pulpito.ceph.com/teuthology-2023-06-02_14:22:10-upgrade:octopus-x-quincy-distro-default-smithi/7294332/