2024-02-18T07:36:33.016 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-02-18T07:36:33.016 INFO:teuthology.results:subset: '111/120000' 2024-02-18T07:36:33.016 INFO:teuthology.results:seed: '5238' 2024-02-18T08:17:07.147 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-02-18T08:17:07.157 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2024-02-18T11:27:21.861 INFO:teuthology.results:Tests finished! gathering results... 2024-02-18T11:27:24.353 INFO:scrape:Found 29 jobs 2024-02-18T11:27:25.334 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565633/teuthology.log 2024-02-18T11:27:31.031 INFO:scrape:Found 3 distinct failure reasons 2024-02-18T11:27:31.031 INFO:scrape:Failure: "2024-02-18T08:26:57.282459+0000 mon.a (mon.0) 642 : cluster [WRN] Health check failed: 1/433 objects unfound (0.231%) (OBJECT_UNFOUND)" in cluster log 2024-02-18T11:27:31.031 INFO:scrape:10 jobs: ['7565628', '7565623', '7565631', '7565617', '7565610', '7565624', '7565611', '7565612', '7565627', '7565618'] 2024-02-18T11:27:31.031 INFO:scrape:suites intersection: [] 2024-02-18T11:27:31.031 INFO:scrape:suites union: ['0-roles', '0-start', '1-ceph-install/octopus', '1-start', '1-tasks', '1.1-pg-log-overrides/short_pg_log', '2-first-half-tasks/radosbench', '2-first-half-tasks/rbd_api', '2-workload/{rgw_ragweed_prepare}', '3-stress-tasks/{radosbench', '3-upgrade-sequence/upgrade-all', '4-pacific', '4-second-half-tasks/radosbench', '4-second-half-tasks/rbd-import-export', '5-final-workload/{rgw', 'centos_latest}', 'mon_election/classic', 'mon_election/classic}', 'mon_election/connectivity', 'mon_election/connectivity}', 'rados_api', 'rados_loadgenbig', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'rbd_import_export', 'readwrite', 'rgw_ragweed_check}', 'snaps-few-objects}', 'start}', 'test_rbd_api}}', 'upgrade-sequence', 'upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack', 'upgrade:octopus-x/parallel/{0-distro/centos_8.stream_container_tools', 'upgrade:octopus-x/parallel/{0-distro/rhel_8.6_container_tools_3.0', 'upgrade:octopus-x/parallel/{0-distro/rhel_8.6_container_tools_rhel8', 'upgrade:octopus-x/parallel/{0-distro/ubuntu_18.04', 'upgrade:octopus-x/parallel/{0-distro/ubuntu_20.04', 'upgrade:octopus-x/stress-split/{0-distro/rhel_8.6_container_tools_3.0', 'upgrade:octopus-x/stress-split/{0-distro/ubuntu_18.04', 'workload/{ec-rados-default'] 2024-02-18T11:27:31.032 INFO:scrape: 2024-02-18T11:27:31.032 INFO:scrape:Dead: None 2024-02-18T11:27:31.032 INFO:scrape:1 jobs: ['7565633'] 2024-02-18T11:27:31.032 INFO:scrape: 2024-02-18T11:27:31.032 INFO:scrape:Failure: Command failed on smithi064 with status 95: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:octopus shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8359654-ce33-11ee-95ba-87774f69a715 -e sha1=81bd20d634209c7cb82c18be12b4b5a05643ebf1 -- bash -c 'ceph orch ps'" 2024-02-18T11:27:31.032 INFO:scrape:7 jobs: ['7565608', '7565619', '7565625', '7565632', '7565609', '7565616', '7565626'] 2024-02-18T11:27:31.032 INFO:scrape:suites intersection: [] 2024-02-18T11:27:31.032 INFO:scrape:suites union: ['0-roles', '0-start', '1-start', '1-tasks', '2-first-half-tasks/rbd-cls', '2-first-half-tasks/rbd-import-export', '2-first-half-tasks/readwrite', '2-first-half-tasks/snaps-few-objects', '3-stress-tasks/{radosbench', '4-second-half-tasks/radosbench', '4-second-half-tasks/rbd-import-export', 'mon_election/classic', 'mon_election/classic}', 'mon_election/connectivity}', 'rados_api', 'rados_loadgenbig', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'rbd_import_export', 'readwrite', 'snaps-few-objects}', 'test_rbd_api}}', 'upgrade-sequence', 'upgrade:octopus-x/parallel/{0-distro/ubuntu_18.04', 'upgrade:octopus-x/parallel/{0-distro/ubuntu_20.04', 'upgrade:octopus-x/stress-split/{0-distro/centos_8.stream_container_tools', 'upgrade:octopus-x/stress-split/{0-distro/rhel_8.6_container_tools_rhel8', 'upgrade:octopus-x/stress-split/{0-distro/ubuntu_18.04', 'upgrade:octopus-x/stress-split/{0-distro/ubuntu_20.04', 'workload/{ec-rados-default'] 2024-02-18T11:27:31.032 INFO:scrape: 2024-02-18T11:27:31.034 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi ================================================================= info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/ failed: 17 dead: 1 running: 0 waiting: 0 queued: 0 passed: 11 Fail ================================================================= [7565608] upgrade:octopus-x/stress-split/{0-distro/rhel_8.6_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} ----------------------------------------------------------------- time: 00:27:01 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565608/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565608/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e494877cb9c848d4a272191b73834196 Command failed on smithi064 with status 95: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:octopus shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e8359654-ce33-11ee-95ba-87774f69a715 -e sha1=81bd20d634209c7cb82c18be12b4b5a05643ebf1 -- bash -c 'ceph orch ps'" [7565609] upgrade:octopus-x/parallel/{0-distro/ubuntu_18.04 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} ----------------------------------------------------------------- time: 00:21:53 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565609/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565609/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2bdcf0c0a44b498e9f717b84958dcdf8 Command failed on smithi131 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:octopus shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7fbf4f4-ce32-11ee-95ba-87774f69a715 -e sha1=81bd20d634209c7cb82c18be12b4b5a05643ebf1 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' [7565610] upgrade:octopus-x/parallel/{0-distro/ubuntu_20.04 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} ----------------------------------------------------------------- time: 01:09:30 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565610/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565610/ "2024-02-18T07:59:34.437835+0000 mon.a (mon.0) 188 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log [7565611] upgrade:octopus-x/stress-split/{0-distro/ubuntu_18.04 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} ----------------------------------------------------------------- time: 02:06:10 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565611/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565611/ "2024-02-18T07:58:26.622027+0000 mon.a (mon.0) 253 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log [7565612] upgrade:octopus-x/parallel/{0-distro/centos_8.stream_container_tools 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} ----------------------------------------------------------------- time: 01:06:37 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565612/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565612/ "2024-02-18T08:03:01.556939+0000 mon.a (mon.0) 260 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log [7565616] upgrade:octopus-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} ----------------------------------------------------------------- time: 00:23:19 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565616/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565616/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c4cefd6dff0c4cd08342e5f8eeb6290f Command failed on smithi135 with status 95: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:octopus shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 668c381e-ce34-11ee-95ba-87774f69a715 -e sha1=81bd20d634209c7cb82c18be12b4b5a05643ebf1 -- bash -c 'ceph orch ps'" [7565617] upgrade:octopus-x/parallel/{0-distro/rhel_8.6_container_tools_3.0 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} ----------------------------------------------------------------- time: 01:15:57 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565617/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565617/ "2024-02-18T08:13:57.240721+0000 mon.a (mon.0) 295 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log [7565618] upgrade:octopus-x/parallel/{0-distro/rhel_8.6_container_tools_rhel8 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} ----------------------------------------------------------------- time: 01:11:18 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565618/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565618/ "2024-02-18T08:14:21.277783+0000 mon.a (mon.0) 328 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log [7565619] upgrade:octopus-x/stress-split/{0-distro/centos_8.stream_container_tools 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} ----------------------------------------------------------------- time: 00:22:50 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565619/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565619/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2410ec1db48840d98fdc220e86f72b27 Command failed on smithi103 with status 95: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:octopus shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1874576e-ce35-11ee-95ba-87774f69a715 -e sha1=81bd20d634209c7cb82c18be12b4b5a05643ebf1 -- bash -c 'ceph orch ps'" [7565623] upgrade:octopus-x/parallel/{0-distro/ubuntu_18.04 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} ----------------------------------------------------------------- time: 01:06:54 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565623/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565623/ "2024-02-18T08:12:27.171667+0000 mon.a (mon.0) 188 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log [7565624] upgrade:octopus-x/stress-split/{0-distro/rhel_8.6_container_tools_3.0 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} ----------------------------------------------------------------- time: 02:18:01 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565624/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565624/ "2024-02-18T08:21:23.076152+0000 mon.a (mon.0) 199 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log [7565625] upgrade:octopus-x/parallel/{0-distro/ubuntu_20.04 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} ----------------------------------------------------------------- time: 00:22:03 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565625/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565625/ sentry: https://sentry.ceph.com/organizations/ceph/?query=33c57b686a2f4b6ca0ffd173eb23680a Command failed on smithi001 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:octopus shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f526118-ce35-11ee-95ba-87774f69a715 -e sha1=81bd20d634209c7cb82c18be12b4b5a05643ebf1 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' [7565626] upgrade:octopus-x/stress-split/{0-distro/rhel_8.6_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} ----------------------------------------------------------------- time: 00:28:23 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565626/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565626/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bd2c24ade20047e4bc89494a9e80793f Command failed on smithi029 with status 95: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:octopus shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 54dcff66-ce36-11ee-95ba-87774f69a715 -e sha1=81bd20d634209c7cb82c18be12b4b5a05643ebf1 -- bash -c 'ceph orch ps'" [7565627] upgrade:octopus-x/parallel/{0-distro/centos_8.stream_container_tools 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} ----------------------------------------------------------------- time: 01:07:29 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565627/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565627/ "2024-02-18T08:16:54.851329+0000 mon.a (mon.0) 258 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log [7565628] upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-workload/{rgw_ragweed_prepare} 3-upgrade-sequence/upgrade-all 4-pacific 5-final-workload/{rgw rgw_ragweed_check} centos_latest} ----------------------------------------------------------------- time: 00:23:34 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565628/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565628/ "2024-02-18T08:26:57.282459+0000 mon.a (mon.0) 642 : cluster [WRN] Health check failed: 1/433 objects unfound (0.231%) (OBJECT_UNFOUND)" in cluster log [7565631] upgrade:octopus-x/parallel/{0-distro/rhel_8.6_container_tools_3.0 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} ----------------------------------------------------------------- time: 01:14:03 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565631/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565631/ "2024-02-18T08:42:55.018244+0000 mon.a (mon.0) 292 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log [7565632] upgrade:octopus-x/stress-split/{0-distro/ubuntu_18.04 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} ----------------------------------------------------------------- time: 00:22:00 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565632/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565632/ sentry: https://sentry.ceph.com/organizations/ceph/?query=07f7ed11579b43f9b1b75f52cbaa3254 Command failed on smithi133 with status 95: "sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:octopus shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37c14f7a-ce38-11ee-95ba-87774f69a715 -e sha1=81bd20d634209c7cb82c18be12b4b5a05643ebf1 -- bash -c 'ceph orch ps'" Dead ================================================================= [7565633] upgrade:octopus-x/parallel/{0-distro/rhel_8.6_container_tools_rhel8 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565633/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565633/ Error reimaging machines: Failed to power on smithi187 Pass ================================================================= [7565605] upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-workload/{rgw_ragweed_prepare} 3-upgrade-sequence/upgrade-mon-osd-mds 4-pacific 5-final-workload/{rgw rgw_ragweed_check} centos_latest} time: 00:31:25 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565605/ [7565606] upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-hybrid 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-pacific 7-final-workload mon_election/connectivity thrashosds-health ubuntu_18.04} time: 01:36:07 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565606/ [7565607] upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-pacific 8-final-workload/{rbd-python snaps-many-objects} mon_election/classic objectstore/filestore-xfs thrashosds-health ubuntu_18.04} time: 02:32:31 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565607/ [7565613] upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-workload/{rgw_ragweed_prepare} 3-upgrade-sequence/upgrade-all 4-pacific 5-final-workload/{rgw rgw_ragweed_check} centos_latest} time: 00:25:09 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565613/ [7565614] upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-hybrid 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-pacific 7-final-workload mon_election/classic thrashosds-health ubuntu_18.04} time: 01:41:29 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565614/ [7565615] upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-pacific 8-final-workload/{rbd-python snaps-many-objects} mon_election/classic objectstore/bluestore-hybrid thrashosds-health ubuntu_18.04} time: 03:07:16 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565615/ [7565620] upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-workload/{rgw_ragweed_prepare} 3-upgrade-sequence/upgrade-mon-osd-mds 4-pacific 5-final-workload/{rgw rgw_ragweed_check} centos_latest} time: 00:30:22 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565620/ [7565621] upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-pacific 7-final-workload mon_election/connectivity thrashosds-health ubuntu_18.04} time: 01:43:07 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565621/ [7565622] upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-pacific 8-final-workload/{rbd-python snaps-many-objects} mon_election/connectivity objectstore/filestore-xfs thrashosds-health ubuntu_18.04} time: 02:34:44 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565622/ [7565629] upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-pacific 7-final-workload mon_election/classic thrashosds-health ubuntu_18.04} time: 01:24:13 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565629/ [7565630] upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-pacific 8-final-workload/{rbd-python snaps-many-objects} mon_election/connectivity objectstore/bluestore-hybrid thrashosds-health ubuntu_18.04} time: 03:05:10 info: https://pulpito.ceph.com/yuriw-2024-02-17_17:35:25-upgrade:octopus-x-pacific-release-distro-default-smithi/7565630/