github-featured / comet-ml--opik

发布

0 发布 8,340 Git 标签

RSS 订阅源
  • 2.0.0-5001 d50f6b8501
    比较

    [OPIK-5691] [BE][FE] feat: add eval suite analytics events (#6326)

    frostbyte_neo 发布于 2026-04-17 09:59:24 +00:00

    • [OPIK-5691] [BE][FE] feat: add eval suite analytics events

    Add three analytics events for the Eval Suite launch dashboards
    tracked in OPIK-5245:

    Backend:

    • opik_eval_suite_created: fired when a TEST_SUITE dataset is created
      (DatasetsResource), capturing both UI and SDK creation paths.
    • opik_eval_suite_run: fired when an experiment is created against a
      TEST_SUITE dataset (ExperimentService), capturing reuse and run
      frequency. Wrapped in try/catch to avoid disrupting experiment
      creation if analytics fails.

    Frontend:

    • opik_eval_suite_ui_configured: fired from the creation dialog
      (useDatasetForm) with UI-specific properties (CSV upload,
      assertions, runs_per_item) not available at the BE endpoint.

    Note: item_count was omitted from opik_eval_suite_run because the
    dataset enrichment needed to fetch it is too heavy for an analytics
    side-effect. Can be added later with a lightweight count query if the
    "test cases per suite" metric proves critical.

    • fix(test): add AnalyticsService mock to test constructors

    DatasetsResourceIntegrationTest and ExperimentServiceTest manually
    construct the resource/service with explicit args — add the missing
    AnalyticsService mock parameter to match the updated constructors.

    • fix(analytics): add opik_ prefix to BE event names

    Event names in code should already include the opik_ prefix per the
    analytics-instrumentation skill doc, even though AnalyticsService
    auto-prepends it at runtime. Aligns BE naming with the FE convention.

    • fix(analytics): offload blocking dataset lookup to boundedElastic scheduler

    Wraps the datasetService.getById() call in trackEvalSuiteRunIfApplicable
    with Schedulers.boundedElastic().schedule() to prevent thread starvation
    in the doOnSuccess callback.

    • fix(analytics): pass userName explicitly to offloaded eval_suite_run event

    The trackEvalSuiteRunIfApplicable call runs on a boundedElastic thread
    outside the request scope, so resolveIdentity() would fall back to the
    anonymous installation ID. Capture userName before scheduling and use the
    explicit-identity trackEvent overload to ensure PostHog funnels work.

    下载附件