发布

  • fix: optimizer fixes and benchmarks working (#2031)

    frostbyte_neo 发布于 2025-05-06 11:13:05 +00:00

    • fix: missing dspy settings

    • fix: missing experiment config for metaprompter

    • fix: benchmarks mapped to latest api

    • feat: universal litellm cache for optimizer

    • feat: added new datasets

    • feat: added test mode and args for benchmark dataset runs

    • fix: cache for test mode runs

    • Update .gitignore

    • chore: responses for dataset outputs at the end

    • fix: demo examples fail when int's

    • feat: various changes to benchmark runs

    • refactor: changes to avoid fork

    as per @dsblank comments/review

    • fix: type mapping for evaluation.metrics.llm_judges.context_precision.metric

    • Update run_benchmark.py

    • fix: ContextPrecision on optimizers

    • fix: opik optimizer truthful_qa

    • fix: opik optimizer ContextPrecision metric mapping

    • Update callback.py

    • Update callback.py

    • Update cache_config.py


    Co-authored-by: Douglas Blank doug.blank@gmail.com

    下载附件