2023-09-01T14:24:07.652 INFO:root:teuthology version: 0.0.1.dev192+gf9a8556 2023-09-01T14:24:07.653 INFO:teuthology.results:subset: '111/120000' 2023-09-01T14:24:07.653 INFO:teuthology.results:seed: '5363' 2023-09-01T14:24:11.285 INFO:root:teuthology version: 0.0.1.dev192+gf9a8556 2023-09-01T14:24:11.286 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-09-02T02:25:02.878 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds 2023-09-02T02:25:03.773 INFO:scrape:Found 9 jobs 2023-09-02T02:25:03.802 INFO:scrape:Found 1 distinct failure reasons 2023-09-02T02:25:03.803 INFO:scrape:Dead: b'2023-09-02T02:25:02.763 INFO:journalctl@ceph.prometheus.a.smithi099.stdout:Sep 02 02:25:02 smithi099 conmon[120146]: ts=2023-09-02T02:25:02.315Z caller=manager.go:609 level=warn component="rule manager" group=pools msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on(pool_id) group_right()\\n ceph_pool_metadata) >= 95\\nlabels:\\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\\n severity: warning\\n type: ceph_default\\nannotations:\\n description: Pool \'{{ $labels.name }}\' will be full in less than 5 days assuming\\n the average fill-up rate of the past 48 hours.\\n summary: Pool growth rate may soon exceed capacity\\n" err="found duplicate series for the match group {pool_id=\\"1\\"} on the left hand-side of the operation: [{instance=\\"smithi099.front.sepia.ceph.com:9283\\", job=\\"ceph\\", pool_id=\\"1\\"}, {instance=\\"172.21.15.84:9283\\", job=\\"ceph\\", pool_id=\\"1\\"}];many-to-many matching not allowed: matching labels must be unique on one side"' 2023-09-02T02:25:03.804 INFO:scrape:1 jobs: ['7386103'] 2023-09-02T02:25:03.804 INFO:scrape:suites: ['0-roles', '1-start', '2-first-half-tasks/rbd-import-export', '3-stress-tasks/{radosbench', '4-second-half-tasks/rbd-import-export', 'mon_election/classic}', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}', 'upgrade:pacific-x/stress-split/{0-distro/rhel_8.4_container_tools_3.0'] 2023-09-02T02:25:03.804 INFO:scrape: 2023-09-02T02:25:03.805 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2023-09-01_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2023-09-01_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2023-09-01_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/ failed: 0 dead: 0 running: 1 waiting: 0 queued: 0 passed: 8 Running ================================================================= [7386103] upgrade:pacific-x/stress-split/{0-distro/rhel_8.4_container_tools_3.0 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} info: http://pulpito.ceph.com/teuthology-2023-09-01_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/7386103/ Pass ================================================================= [7386098] upgrade:pacific-x/stress-split/{0-distro/rhel_8.4_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:18:26 info: http://pulpito.ceph.com/teuthology-2023-09-01_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/7386098/ [7386099] upgrade:pacific-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:23:59 info: http://pulpito.ceph.com/teuthology-2023-09-01_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/7386099/ [7386100] upgrade:pacific-x/stress-split/{0-distro/centos_8.stream_container_tools 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:02:25 info: http://pulpito.ceph.com/teuthology-2023-09-01_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/7386100/ [7386101] upgrade:pacific-x/parallel/{0-random-distro$/{ubuntu_20.04} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:30:58 info: http://pulpito.ceph.com/teuthology-2023-09-01_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/7386101/ [7386102] upgrade:pacific-x/stress-split/{0-distro/centos_8.stream_container_tools_crun 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 01:57:42 info: http://pulpito.ceph.com/teuthology-2023-09-01_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/7386102/ [7386104] upgrade:pacific-x/stress-split/{0-distro/rhel_8.4_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:08:33 info: http://pulpito.ceph.com/teuthology-2023-09-01_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/7386104/ [7386105] upgrade:pacific-x/parallel/{0-random-distro$/{ubuntu_20.04} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:32:27 info: http://pulpito.ceph.com/teuthology-2023-09-01_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/7386105/ [7386106] upgrade:pacific-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:31:35 info: http://pulpito.ceph.com/teuthology-2023-09-01_14:23:02-upgrade:pacific-x-quincy-distro-default-smithi/7386106/