github-featured / comet-ml--opik

发布

0 发布 8,340 Git 标签

RSS 订阅源
  • 1.9.1-3221 2f9b605289
    比较

    [OPIK-2496] [BE] [SDK] Add provider info to token usage section in traces metadata (#3731)

    frostbyte_neo 发布于 2025-11-06 14:56:25 +00:00

    • [OPIK-2496] [FE] Add provider info to token usage section in traces metadata

    • Revision 2: Add provider aggregation to trace spans_agg queries and Trace model

    • Revision 3: Update frontend to display providers array for traces

    • Revision 3: Add comprehensive tests for providers field aggregation

    • Revision 4: Add providers field to trace filters for filtering by provider

    • Revision 5: Add filtering tests for PROVIDERS field (contains and not_contains operators)

    • Refactor TraceDAO to always include providers field in query results

    • Remove providers field from FilterQueryBuilder aggregation settings

    • Enhance TraceDAO to include provider information in span queries

    • Added providers field to span queries, utilizing arraySort(groupUniqArrayIf(provider, provider != '')) to filter out empty provider entries.
    • Updated relevant SELECT statements to include the new provider field in the results.
    • Removed redundant binding of providers in the statement preparation.

    This change improves the trace data by ensuring that only valid provider information is captured and returned in the span queries.

    • Add providers field to TraceTreeViewer tests
    • Updated the test file for TraceTreeViewer to include a new providers field in the mock data, enhancing the test coverage for scenarios involving multiple providers.
    • This change aligns with recent updates to span queries that incorporate provider information, ensuring tests reflect the latest data structure.

    • Revision 2: Fix providers field in tests - update updateSpanCounts helper

    • Refactor Trace to use List for providers field

    • Updated the providers field in the Trace record to use List<String> instead of String[], enhancing type safety and flexibility.
    • Adjusted the TraceDAOImpl to retrieve providers as a List from the database.
    • Modified tests in TracesResourceTest to accommodate the new List type for providers, ensuring consistency across the codebase.
    • Update TraceAssertions to include 'providers' in ignored fields
    • Added 'providers' to the IGNORED_FIELDS_TRACES array in TraceAssertions.java to ensure it is excluded from trace comparisons in tests, aligning with recent updates to the Trace structure.
    • Refactor TraceAssertions to maintain consistency in ignored fields
    • Adjusted the formatting of the IGNORED_FIELDS_TRACES array in TraceAssertions.java for improved readability, ensuring alignment with recent updates while keeping 'providers' as an ignored field.
    • Add support for LIST type in FilterQueryBuilder
    • Enhanced the FilterQueryBuilder to include a new handling case for the LIST field type, allowing for the use of the arrayExists function in filter queries. This addition improves the flexibility of query building for list-based fields.

    • Remove broken providers tests, in TracesResourceTest.java

    • Add comprehensive tests for trace providers in TracesResourceTest.java

    • Introduced multiple test cases to validate the behavior of trace providers, ensuring they reflect unique, non-empty provider names sorted alphabetically.
    • Added scenarios to check for empty providers when no spans are present or when all spans have empty provider strings.
    • Implemented tests to verify that each trace correctly aggregates providers from its associated spans and that filtering by provider works as expected.
    • Update test command syntax for consistency across documentation
    • Changed test command examples in create-and-run-tests.md, test-workflow.mdc, and testing.mdc to use quotes around test class and method names for improved clarity and consistency.

    • Remove 'providers' references from Field and TraceField interfaces, and update FilterQueryBuilder accordingly. Clean up related tests in TracesResourceTest to reflect these changes, enhancing code clarity and consistency.

    • Remove LIST type handling from FilterQueryBuilder to streamline query construction. This change simplifies the logic for filtering operations, enhancing code clarity and maintainability.

    • Add handling for 'providers' field in EXCLUDE_FUNCTIONS of TracesResourceTest

    • Updated the EXCLUDE_FUNCTIONS map to include a case for the 'providers' field, allowing for null values in test scenarios. This change enhances the test coverage and ensures consistency with recent updates to the TraceField interface.

    • Remove 'providers' aggregation from TraceDAOImpl to streamline query results. This change enhances clarity and aligns with recent updates to the TraceField interface.

    • Refactor TracesResourceTest to replace batchCreateTraces with createTrace method

    • Updated multiple instances in TracesResourceTest.java to use the createTrace method instead of batchCreateTraces, enhancing clarity and aligning with recent changes in trace creation logic.

    • Revision 3: Refactor providers tests into parameterized test

    • Spreading should be before setting a new value, not the other way around

    • Enhance metadata handling by injecting provider information into Span and Trace DTOs. Updated JsonUtils to support injecting single and multiple providers into metadata. Refactored related mapping logic in SpanDAO and TraceDAO. Adjusted frontend component to directly use usage data without additional processing.

    • Refactor comments in SpanDAO, TraceDAO, and JsonUtils for clarity on provider metadata injection. Adjusted descriptions to enhance understanding of metadata handling processes.

    • Refactor JsonUtils to streamline provider metadata injection. Introduced helper methods for injecting single and multiple providers, enhancing code clarity and maintainability. Updated comments for better understanding of metadata handling processes.

    • Refactor metadata injection methods in JsonUtils for SpanDAO and TraceDAO. Updated to use new helper methods for injecting single and array fields, improving code clarity and maintainability. Adjusted related mapping logic in both DAOs to reflect these changes.

    • Refactor metadata handling in SpanDAO and TraceDAO to improve clarity. Updated logic for injecting provider and providers metadata, ensuring base metadata is processed before injection. Adjusted comments for better understanding of the mapping process.

    • Remove providers field from Trace interface to streamline metadata structure and improve clarity in type definitions.

    • Update Trace metadata structure in tests to include provider information in the metadata field, enhancing clarity and consistency in test cases.

    • Revision 2: Add comprehensive tests for provider/providers in metadata

    • Revision 3: Refactor metadata injection into private helper methods

    • Revision 4: Extract lambda body into named methods to minimize git diff

    • Revision 5: Fix frontend test to expect providers in metadata

    • Fix type safety in TraceDAO by adding a suppression annotation for unchecked cast on providers list retrieval.

    • Revision 7: Add comprehensive metadata tests for both traces and spans

    • Revision 8: Fix Python SDK test verifiers to allow backend-injected metadata fields

    • Fix formatting in helpers.test.ts

    • Revision 9: Simplify metadata subset check using Python dict operations

    • Revision 10: Revert verifiers to exact equality checks and update tests to include expected provider/providers in metadata

    • Revision 11: Add OpenAPI documentation for provider/providers injection in metadata

    • Revision 12: Clarify metadata description to reflect backend-injected fields

    • Revision 13: Add test helpers to inject provider/providers into metadata for assertions

    • Revision 14: Fix metadata assertions for spans with providers (part 1 - 19 tests fixed)

    • Revision 15: Fix all SpansResourceTest by preparing spans in assertion helpers

    • Revision 16: Fix updateOnlyMetadata test to expect provider in metadata

    • Revision 17: Fix image truncation tests to expect provider in metadata

    • Revision 18: Fix TracesResourceTest to prepare traces with providers in metadata

    • Revision 19: Fix remaining SpansResourceTest helper methods to prepare spans

    • Revision 20: Fix Stats and Stream test assertions to prepare spans

    • Revision 21: Fix TraceStreamTestAssertion to prepare traces

    • Revision 22: Backend Tests timeout, increase from 35 to 40

    • Revision 24: Use actual providers when preparing expected traces/spans for assertions

    • Revision 25: Fix getAndAssert to use actual providers for assertions

    • Revision 26: Enhance assertion helpers to automatically handle provider matching

    • Revision 27: Fix stats calculation to use raw database data without provider injection

    • Revision 28: Remove manual provider injection from test helpers - now handled automatically

    • Revision 29: Make prepare*ForAssertion methods private to enforce proper testing pattern

    • Revision 30: Remove manual prepareSpanForAssertion call from SpansResourceTest

    • Revision 31: Remove all manual prepareSpansForAssertion calls from SpansResourceTest

    • Revision 32: Remove all manual prepareTracesForAssertion calls from TracesResourceTest

    • Undo changes to TraceTreeViewer tests, as logic is only in BE, and the tests mock other logic

    • Revision 18: Fix findWithImageTruncation tests to use assertSpan/assertTraces

    The findWithImageTruncation tests in both SpansResourceTest and TracesResourceTest
    were using direct assertThat() instead of the assertion helpers, bypassing the
    automatic provider injection logic. This was causing test failures because the
    actual spans/traces from the database had provider info injected into metadata,
    but the expected spans/traces did not.

    Fixed by replacing direct assertThat() calls with assertSpan() and assertTraces()
    which automatically handle provider injection for accurate comparison.

    • refactor(tests): Simplify span assertion in FindSpansResourceTest, after merge conflict

    • refactor(tests): Remove outdated comments regarding provider injection in span and trace assertions

    • [OPIK-2496] [P SDK] Update metadata in langchain test to include provider info

    • Modified the metadata in the langchain integration test to include a list of providers, enhancing the clarity of the test's context.
    • [OPIK-2496] [P SDK] Remove provider info from metadata in langchain test
    • Updated the metadata in the langchain integration test by removing the list of providers, simplifying the test's context.

    • Update metadata in E2E tracing tests to clarify provider information injection

    • Revert manual changes to auto-generated openapi.yaml

    • Refactor JsonUtils to use Apache Commons utilities for null and empty checks in metadata injection methods

    • Add @NonNull validation to JsonUtils metadata injection methods

    • Refactor: Rename inject* methods to prepend* and remove comments

    • Renamed injectStringFieldIntoMetadata to prependStringFieldIntoMetadata
    • Renamed injectArrayFieldIntoMetadata to prependArrayFieldIntoMetadata
    • Renamed injectFieldIntoMetadata to prependFieldIntoMetadata
    • Renamed copyMetadataFields to copyJsonNode
    • Removed comments from all renamed methods
    • Updated all references across SpanDAO, TraceDAO, test files
    • Refactor: Simplify method names and parameters in JsonUtils
    • Renamed prependFieldIntoMetadata to prependField
    • Renamed metadata parameter to jsonNode for clarity
    • Refactor: Use method overloading for prependField methods
    • Renamed prependStringFieldIntoMetadata to prependField(String)
    • Renamed prependArrayFieldIntoMetadata to prependField(List)
    • Now using Java method overloading with 3 variants:
      • prependField(JsonNode, String, String)
      • prependField(JsonNode, String, List)
      • prependField(JsonNode, String, JsonNode) [private]
    • Updated all references across DAO and test files
    下载附件