数据集 / cais/mmlu

cais/mmlu 已完整同步

annotations_creators:

  • no-annotation language_creators:
  • expert-generated language:
  • en license:
  • mit multilinguality:
  • monolingual size_categories:
  • 10K<n<100K source_datasets:
  • original task_categories:
  • question-answering task_ids:
  • multiple-choice-qa paperswithcode_id: mmlu pretty_name: Measuring Massive Multitask Language Understanding language_bcp47:
  • en-US dataset_info:
  • config_name: abstract_algebra features:
    • name: question dtype: string
    • name: subject dtype: string
    • name: choices sequence: string
    • name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits:
    • name: test num_bytes: 49618.6654322746 num_examples: 100
    • name: validation num_bytes: 5485.515349444808 num_examples: 11
    • name: dev num_bytes: 2199.1754385964914 num_examples: 5 download_size: 17143 dataset_size: 57303.3562203159
  • config_name: all features:
    • name: question dtype: string
    • name: subject dtype: string
    • name: choices sequence: string
    • name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits:
    • name: test num_bytes: 6967453 num_examples: 14042
    • name: validation num_bytes: 763484 num_examples: 1531
    • name: dev num_bytes: 125353 num_examples: 285
    • name: auxiliary_train num_bytes: 161000625 num_examples: 99842 download_size: 51503402 dataset_size: 168856915
  • config_name: anatomy features:
    • name: question dtype: string
    • name: subject dtype: string
    • name: choices sequence: string
    • name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits:
    • name: test num_bytes: 66985.19833357072 num_examples: 135
    • name: validation num_bytes: 6981.5649902024825 num_examples: 14
    • name: dev num_bytes: 2199.1754385964914 num_examples: 5 download_size: 28864 dataset_size: 76165.9387623697
  • config_name: astronomy features:
    • name: question dtype: string
    • name: subject dtype: string
    • name: choices sequence: string
    • name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits:
    • name: test num_bytes: 75420.3714570574 num_examples: 152
    • name: validation num_bytes: 7978.931417374265 num_examples: 16
    • name: dev num_bytes: 2199.1754385964914 num_examples: 5 download_size: 39316 dataset_size: 85598.47831302814
  • config_name: auxiliary_train features:
    • name: train struct:
      • name: answer dtype: int64
      • name: choices sequence: string
      • name: question dtype: string
      • name: subject dtype: string splits:
    • name: train num_bytes: 161000625 num_examples: 99842 download_size: 47518592 dataset_size: 161000625
  • config_name: business_ethics features:
    • name: question dtype: string
    • name: subject dtype: string
    • name: choices sequence: string
    • name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits:
    • name: test num_bytes: 49618.6654322746 num_examples: 100
    • name: validation num_bytes: 5485.515349444808 num_examples: 11
    • name: dev num_bytes: 2199.1754385964914 num_examples: 5 download_size: 31619 dataset_size: 57303.3562203159
  • config_name: clinical_knowledge features:
    • name: question dtype: string
    • name: subject dtype: string
    • name: choices sequence: string
    • name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits:
    • name: test num_bytes: 131489.4633955277 num_examples: 265
    • name: validation num_bytes: 14461.813193990856 num_examples: 29
    • name: dev num_bytes: 2199.1754385964914 num_exam

数据概览

MMLU 面向大语言模型多任务知识评测，当前镜像包含题目、学科、选项和答案相关文件，适合做基准测试与能力分析。

181 个文件

浏览文件

数据集版权信息

本数据集的许可证为 MIT License。如有违反相关条款，请联系 WEHUB，我们将及时处理。 查看许可证