github-featured / comet-ml--opik

发布

0 发布 8,340 Git 标签

RSS 订阅源
  • 1.10.2-4066 1cb281e737
    比较

    [OPIK-3920] [BE][FE] Add experiments column and filter to Traces tab (#4806)

    frostbyte_neo 发布于 2026-02-02 11:17:22 +00:00

    • [OPIK-3920] [BE][FE] Add experiment column and filter to Traces tab

    • Revision 2: Add experiment column sorting support

    • Revision 3: Add experiment_name filter with partial matching support

    • Revision 4: Add Experiment ID column to traces table

    • Revision 5: Add sorting support for Experiment ID column

    • Revision 6: Add ExperimentSelectBox for smart experiment name filter

    • [OPIK-3920] [BE] Remove unused ExperimentDAO code

    • Revision 7: Fix experiment.id sorting - skip dynamic null handling for mapped fields

    • Revision 8: Fix experiment.id sorting - exclude mapped fields from dynamic key binding

    • Revision 9: Simplify experiment_id sorting - use underscore instead of dot to avoid dynamic field treatment

    • Revision 10: Use equals operator for experiment_name filter instead of contains

    • Revision 11: Rename experiment column id to experiment_name for consistency

    • Revision 12: Sort experiments by name in ExperimentSelectBox filter

    • Revision 13: Use experiment_id filter for both name dropdown and ID input

    • Revision 14: Remove table prefix from experiment_id filter field

    • Revision 15: Fix duplicate filter issue by using separate IDs with transform function

    • Revision 16: Fix - dropdown on Experiment name, text input on Experiment ID

    • Revision 17: Remove duplicate customMeta, add transform to stats hook

    • Revision 18: Use ResourceCell for experiment columns, add datasetId to ExperimentReference for navigation

    • Revision 11: Simplify to single Experiment column and filter (remove Experiment ID)

    • Revision 12: Use experiment_id consistently, remove transform function

    • Revision 13: Remove unused overloaded hasDynamicKeys and bindDynamicKeys methods

    • Revision 14: Use anyMatch instead of findAny().isPresent() for better readability

    • Revision 15: Remove unused isMappedField check

    • Revision 16: Use named tuple elements and extract field mapping to static constant

    • Revision 17: Flatten experiment tuple into separate columns for cleaner mapping

    • Revision 18: Extract experiment_id to COLUMN_EXPERIMENT_ID constant in shared types

    • Revision 19: Add experiment to IGNORED_FIELDS_TRACES in test assertions

    • Revision 20: Add comprehensive tests for experiment reference, sorting, and filtering

    • Revision 21: Move FilterOperator and TraceField imports to top of file

    • Revision 22: Fix filter operator to use Operator.EQUAL instead of FilterOperator.EQUALS

    • Revision 23: Add Dataset, Experiment, ExperimentItem imports and remove fully qualified class names

    • Revision 24: Remove remaining fully qualified class names from test code

    • Revision 25: Use TraceFilter.builder() instead of Filter.builder() and pass TraceField directly

    • Revision 26: Extract common test setup and parameterize sorting test

    • Revision 27: Fix test failures - add EXPERIMENT to EXCLUDE_FUNCTIONS and use unique experiment names

    • Revision 28: Add unique suffix to experiment names in filtering test

    • Revision 29: Use separate random suffixes for each experiment in filtering test

    • Revision 30: Use separate random suffixes for each experiment in sorting test

    • Revision 31: Use createPartialExperiment() for all experiment tests to avoid 409 conflicts

    • Revision 32: Remove unused Experiment import

    • Revision 33: Use datasetName instead of datasetId when creating experiments (datasetId is read-only)

    • Revision 35: Add projectId filter to ExperimentsSelectBox

    • Revision 36: Remove unnecessary INNER JOIN and fix deduplication in experiment_filters subquery

    • Revision 37: Address PR review comments - fix deduplication, optimize performance, and improve code quality

    Critical fixes:

    • Revert LIMIT 1 BY trace_id to LIMIT 1 BY id in experiment_filters subqueries (3 locations)
      The correct deduplication for experiment_items is by id, not trace_id
    • Remove unnecessary LIMIT 1 BY ei.trace_id in experiments_agg CTE
      trace_id is part of the primary key and doesn't need deduplication, use DISTINCT instead

    Performance optimizations:

    • Filter experiments early by only selecting experiments referenced in current query scope
      Prevents retrieving all workspace experiments when only a subset is needed
    • Use full primary key ORDER BY (workspace_id, dataset_id, id) in experiments subquery
      Improves ClickHouse query performance

    Code quality improvements:

    • Fix mapExperiment null check to only validate key fields (experimentId, experimentDatasetId)
      experimentName is editable and its absence doesn't indicate missing data
    • Read UUID fields directly from DB instead of String conversion
      Avoids unnecessary UUID.fromString() calls
    • Move EXPERIMENT_FIELD_MAPPING constant to TraceSortingFactory
      Better organization and encapsulation of sorting-related constants

    Addresses comments from andrescrz in PR #4806

    • Revision 38: Remove redundant experiment filtering in experiments_agg CTE

    The id IN (SELECT DISTINCT experiment_id FROM experiment_items ...) filter
    is redundant because the INNER JOIN on ei.experiment_id = e.id already
    ensures we only get experiments that exist in the filtered experiment_items.

    The join naturally limits to only experiments referenced in the ei subquery,
    which is already filtered by workspace_id and time range.

    • Revision 39: Remove unnecessary table alias in experiment_filters subqueries

    Since there's only one table (experiment_items) in these subqueries,
    the ei. alias prefix is unnecessary and makes the code more verbose.

    Simplified across all 3 occurrences of the experiment_filters subquery.

    • Revision 40: Add builder pattern to ExperimentReference and TestContext records

    Added @Builder(toBuilder = true) to both records for better maintainability
    and readability, as suggested in code review.

    Changes:

    • Added @Builder(toBuilder = true) to ExperimentReference record
    • Updated mapExperiment() to use builder pattern instead of constructor
    • Added @Builder(toBuilder = true) to TestContext test record
    • Updated setupWorkspaceProjectDataset() to use builder pattern
    • Added lombok.Builder import to TracesResourceTest

    This makes the code more maintainable and follows the builder pattern
    convention used throughout the codebase.

    • Revision 41: Fix ClickHouse UUID null handling in mapExperiment
    • Read experiment UUID fields as String instead of UUID to handle ClickHouse R2DBC driver quirks
    • Add check for CLICKHOUSE_FIXED_STRING_UUID_FIELD_NULL_VALUE (36 null characters)
    • ClickHouse returns \u0000 repeated 36 times for NULL UUID FixedString fields
    • Convert to UUID using UUID.fromString() after validation
    • This prevents IllegalArgumentException when LEFT JOIN experiments_agg doesn't match
    • Revision 42: Optimize experiments_agg CTE to only build when needed
    • Wrap experiments_agg CTE in conditional: <if(sort_has_experiment || !exclude_experiment)>
    • CTE is now only built when:
      1. Sorting by experiment_id (sort_has_experiment=true), OR
      2. Experiment field is NOT excluded (!exclude_experiment=true)
    • Performance improvement: Skip expensive CTE and JOINs when experiment column is hidden
    • Frontend will pass exclude=experiment when column is not visible
    • Backend already prevents excluding fields used in sorting (line 3015)
    • Revision 42: Add exclude parameter to optimize experiment data fetching

    Frontend changes:

    • Add exclude parameter to useTracesList hook to pass excluded fields to backend
    • Update useTracesOrSpansList to support exclude parameter
    • Compute excludeFields in TracesSpansTab based on visible columns
    • Exclude experiment field when COLUMN_EXPERIMENT_ID is not in selectedColumns
    • Automatically refetch when column visibility changes (via useMemo dependency)

    Backend optimization:

    • Frontend passes exclude=experiment when column is hidden
    • Backend skips building experiments_agg CTE when excluded
    • Significant performance improvement for traces queries without experiment column

    Note: Backend already prevents excluding fields used in sorting (line 3015 in TraceDAO.java)

    • Revision 43: Only exclude experiment field for traces, not spans

    • Revision 44: Fix experiments_agg JOIN in final SELECT to include sorting condition

    • Changed condition from <if(!exclude_experiment)> to <if(sort_has_experiment || !exclude_experiment)>
    • Ensures experiments_agg is joined when sorting by experiment, even if excluded
    • Fixes "Unknown expression identifier eaag.experiment_name" error when sorting by experiment
    • Matches the conditional logic used in traces_final CTE (line 1090)
    下载附件