github-featured / comet-ml--opik

发布

0 发布 8,340 Git 标签

RSS 订阅源
  • 1.8.28-2417 e53b7eadcf
    比较

    [OPIK-2225] [P SDK] Langchain: Failed to extract token usage from presumably OpenAI LLM langchain run. (#3011)

    frostbyte_neo 发布于 2025-08-14 15:27:44 +00:00

    • [OPIK-2225]: Added additional information for usage token extraction to have run_dict printed.

    • [OPIK-2225]: Enhanced LangChain integration for token usage extraction and metadata consistency.

    • Updated LangChain token extraction logic to support updated usage metadata keys (input_tokens, output_tokens, and total_tokens).
    • Introduced try_to_get_usage and find_token_usage_dict functions for modular usage data handling.
    • Refactored related test cases to validate streaming and stream_usage configurations and scenarios.
    • Improved logging messages for better clarity on token usage availability.

    • [OPIK-2225]: Refactored Anthropic usage extraction to use a more robust mechanism.

    • [OPIK-2225]: Refactored LangChain usage helpers for flexible token usage extraction.

    • Renamed try_to_get_usage to try_to_get_usage_by_search for clarity.
    • Updated find_token_usage_dict to support strict key matching logic.
    • Applied changes across Anthropic and OpenAI usage extractors to ensure consistency.
    • [OPIK-2225]: Improved error handling and logging for token usage extraction.
    • Added log_once_at_level warnings for failed usage extractions in Anthropic and Bedrock extractors.
    • Ensured consistent logic for logging and handling exceptions across provider usage extractors.

    • [OPIK-2225]: Fixed to invoke correct method

    • [OPIK-2225]: Enhanced Google GenAI usage extraction logic.

    • Improved try_get_token_usage to handle token usage extraction with updated error logging and warnings.
    • Added conditional checks for LangChainUsage and incorporated Google Gemini mapping for token usage.
    • Updated logging to provide clearer warnings when token usage data is unavailable or extraction fails.
    • [OPIK-2225]: Enhanced token usage extraction for Anthropic and Groq integrations.
    • Streamlined log_once_at_level method calls in Anthropic usage extractor.
    • Improved Groq token extraction by introducing GROQ_CANDIDATE_USAGE_KEYS and updating logic for streaming mode handling.
    • Refined error logging for better clarity when token usage extraction fails.

    • [OPIK-2225]: Updated Groq extractor logging to include more details in error.

    • [OPIK-2225]: Enhanced VertexAI usage extraction and logging.

    • Introduced VERTEXAI_CANDIDATE_USAGE_KEYS for improved token usage identification.
    • Updated _try_get_token_usage logic to support LangChainUsage and Google Gemini mapping.
    • Enhanced error logging with detailed warnings when token usage extraction fails.
    • Added a new test to validate streaming token usage logging in LangChain VertexAI integration.

    • [OPIK-2225]: Updated Anthropic tests to include new ephemeral token usage keys

    • [OPIK-2225]: Fixed test coverage for OpenAI span modeling in dspy integration.

    • Added additional Span with error details.
    • [OPIK-2225]: Updated DSPy integration test to handle version-based span conditions.
    • Added checks for dspy version to conditionally include additional spans in tests.
    • [OPIK-2225]: Adjusted evaluation metric threshold to avoid flaky test failures.
    • Updated assertion in test_evaluation_metrics to include boundary condition for poor trajectory scoring.
    • [OPIK-2225]: Added Groq integration tests and improved configuration checks.
    • Introduced ensure_groq_configured fixture for validating Groq environment settings.
    • Removed redundant logging in Groq usage extractor.
    • Added comprehensive tests for Groq LLM integration, covering token usage logging and trace modeling.

    • [OPIK-2225]: Updated dependencies for LangChain

    • [OPIK-2225]: Fixed GitHub workflow to introduce environment variable GROQ_API_KEY with value taken from secrets.

    • [OPIK-2225]: Corrected method name in Groq usage extractor from map_to_openai_completions_usage to map_to_groq_completions_usage.

    下载附件