github-featured / vibrantlabsai--ragas

发布

0 发布 90 Git 标签

RSS 订阅源
  • v0.1.3 12ad1905f2
    比较

    Add support for optional max concurrency (#643)

    frostbyte_neo 发布于 2024-02-28 03:22:53 +00:00

    Added optional Semaphore-based concurrency control for #642
    As for the default value for max_concurrency, I don't know the ratio
    of API users vs. local LLM users, so the proposed default is an
    opinionated value of 16

    • I think more people use OpenAI API for now vs. local LLMs, thus
      default is not -1 (no limit)
    • 16 seems to be reasonably fast and doesn't seem to hit throughput
      limit in my experience

    Tests
    Embedding for 1k documents finished in <2min and subsequent Testset
    generation for test_size=1000 proceeding without getting stuck:
    image

    another 30s passes:
    image

    Co-authored-by: Jithin James jamesjithin97@gmail.com

    下载附件