github-featured / comet-ml--opik

发布

0 发布 8,340 Git 标签

RSS 订阅源
  • 1.9.25-3405 7144e24345
    比较

    [OPIK-3181] [BE] Fix Redis codec to respect configured JSON string limit (#4157)

    frostbyte_neo 发布于 2025-11-21 11:36:50 +00:00

    • [OPIK-3181] [BE] Fix Redis codec to respect configured JSON string limit

    Implemented lazy codec initialization to ensure Redisson codecs use the
    configured string length limit instead of the 20MB default.

    Root cause: Static codec initialization in OnlineScoringCodecs, WebhookConfig,
    and TraceThreadConfig was capturing JsonUtils.getMapper() before the application
    configured it with correct value in OpikApplication.run(). This caused Redis deserialization
    to fail with JsonMappingException for payloads >20MB.

    Solution: Changed static codec fields to use Supplier pattern (OnlineScoringCodecs)
    or factory methods (WebhookConfig, TraceThreadConfig) to defer codec creation until
    after JsonUtils.configure() has set the 100MB limit.

    Changes:

    • OnlineScoringCodecs: Use Supplier for lazy initialization
    • WebhookConfig: Replace static CODEC with createCodec() factory method
    • TraceThreadConfig: Replace static CODEC with createCodec() factory method
    • JsonUtils: Use StreamReadConstraints.DEFAULT_MAX_STRING_LEN constant

    Tests:

    • OnlineScoringCodecsTest: Unit tests for lazy codec initialization
    • OnlineScoringCodecsRedisIntegrationTest: Integration tests with actual Redis
    • Verified with 50MB trace payload via REST API (succeeds with fix, fails without)
    • Revision 2: Rename OnlineScoringCodecs to RedisStreamCodec and move to infrastructure/redis

    Refactored OnlineScoringCodecs -> RedisStreamCodec to better reflect its purpose
    and location. This enum is used throughout Redis stream configurations, not just
    for online scoring, so it belongs in the infrastructure/redis package.

    Changes:

    • Renamed OnlineScoringCodecs to RedisStreamCodec
    • Moved from api/resources/v1/events/ to infrastructure/redis/
    • Updated all imports across WebhookConfig, TraceThreadConfig, OnlineScorePublisher,
      OnlineScoringStreamConfigurationAdapter, and test files
    • Test files also renamed and moved to infrastructure/redis/
    • Revision 3: Add Guava memoization and consolidate tests
    • Use Suppliers.memoize() for JAVA codec to cache instances and reduce GC pressure
    • RedisStreamCodecIntegrationTest into RedisStreamCodecTest

    Benefits:

    • Memoization: Same instance reused, avoiding repeated object creation
    • Thread-safe: Guava's memoize() handles concurrency
    • Cleaner tests: Single test file with proper lifecycle management
    下载附件