🐋 The OpenOrca Dataset! 🐋

We are thrilled to announce the release of the OpenOrca dataset! This rich collection of augmented FLAN data aligns, as best as possible, with the distributions outlined in the Orca paper. It has been instrumental in generating high-performing model checkpoints and serves as a valuable resource for all NLP researchers and developers!

Official Models

Mistral-7B-OpenOrca

Our latest model, the first 7B to score better overall than all previous models below 30B. 98% of Llama2-70b-chat's performance, in a completely open 7B!

OpenOrca-Platypus2-13B

Our third model, the first 13B model to score higher than LLaMA1-65B on the HuggingFace Leaderboard! Released in partnership with Platypus.

LlongOrca 7B & 13B

Our first 7B release, trained on top of LLongMA2 to achieve 16,000 tokens context. #1 long context 7B model at release time, with >99% of the overall #1 model's performance.
LlongOrca-13B-16k, trained on top of LLongMA2. #1 long context 13B model at release time, with >97% of the overall #1 model's performance.

OpenOrcaxOpenChat-Preview2-13B

Our second model, highlighting that we've surpassed the performance reported in the Orca paper. Was #1 at release time, now surpassed by our own OpenOrca-Platypus2-13B. Released in partnership with OpenChat.

OpenOrca-Preview1-13B

OpenOrca-Preview1-13B This model was trained in less than a day, for <$200, with <10% of our data. At release, it beat the current state of the art models on BigBench-Hard and AGIEval. Achieves ~60% of the improvements reported in the Orca paper.

Dataset Summary

The OpenOrca dataset is a collection of augmented FLAN Collection data. Currently ~1M GPT-4 completions, and ~3.2M GPT-3.5 completions. It is tabularized in alignment with the distributions presented in the ORCA paper and currently represents a partial completion of the full intended dataset, with ongoing generation to expand its scope. The data is primarily used for training and evaluation in the field of natural language processing.

Dataset Attribution

We would like to give special recognition to the following contributors for their significant efforts and dedication:

Teknium                     
WingLian/Caseus
Eric Hartford
NanoBit
Pankaj
Winddude
Rohan

http://AlignmentLab.ai:
Autometa
Entropi
AtlasUnified
NeverendingToast
NanoBit
WingLian/Caseus

Also of course, as always, TheBloke, for being the backbone of th

5 个文件

浏览文件

数据集版权信息

本数据集的许可证为 MIT License。如有违反相关条款，请联系 WEHUB，我们将及时处理。查看许可证

Open-Orca/OpenOrca 已完整同步

Table of Contents