github-featured / comet-ml--opik

发布

0 发布 8,340 Git 标签

RSS 订阅源
  • 1.8.44-2611 9d86c1adb8
    比较

    [OPIK-2286] [P SDK] Implement support for a more granular support for choosing dataset items in the experiment (#3199)

    frostbyte_neo 发布于 2025-09-10 11:39:59 +00:00

    • [OPIK-2286] Add dataset sampling support to the evaluation framework
    • Introduced BaseDatasetSampler and RandomDatasetSampler for modular dataset sampling.
    • Updated evaluation methods to accept an optional dataset sampler for selective sampling logic.
    • Enhanced unit tests with scenarios for random sampling.
    • Added samplers module and integrated it into the evaluation engine.
    • [OPIK-2382] Add random dataset sampling integration tests and assertions in evaluation logic
    • Added test_evaluate_prompt__with_random_sampling__happy_flow to validate selective sampling logic with a 3-sample limit.
    • [OPIK-2286] Add dataset sampler support to task_evaluator.evaluate()
    • Extended task_evaluator.evaluate() to support an optional dataset_sampler argument.
    • Updated logic and docstrings to reflect the inclusion of dataset sampling.

    • [OPIK-2286] Fixed grammatic error

    • [OPIK-2286] Split BaseDatasetSampler and RandomDatasetSampler into separate modules

    • Moved RandomDatasetSampler to a new random_dataset_sampler.py file.
    • Updated imports and __all__ in the samplers package to reflect changes.

    • [OPIK-2286] Add docstring for BaseDatasetSampler.sample() method

    • [OPIK-2286] Fixed imports

    • [OPIK-2286] Removed sampler reference from opik_optimizer until OPIK with samplers will be released and become available to import for other projects.

    • [OPIK-2286] Add documentation for dataset samplers

    • Added usage examples and implementation details for custom dataset samplers in the documentation.
    • [OPIK-2286] Enhanced dataset sampler documentation with filtering example
    • Added a detailed code example showcasing dataset sampling using filtering criteria with regex.
    • Updated the description of custom dataset sampler usage.
    下载附件