github-featured / thu-maic--openmaic

发布

0 发布 73 Git 标签

RSS 订阅源
  • @openmaic/generation@0.3.7 9fe650d994
    比较

    fix(quiz): resolve AI answer keys to option values before grading (#1328)

    frostbyte_neo 发布于 2026-09-10 21:28:20 +00:00

    • fix(quiz): resolve AI answer keys to option values before grading

    AI-generated quiz answer keys are unreliable: the model may write the
    correct answer as option CONTENT ("(6, 2)"), as a LETTER ("A"), or as a
    formatting variant (full-width chars, extra spaces) of either. Grading
    compares exact option values, so a learner picking the correct option
    was still graded incorrect whenever the key was written as content or
    in a different format.

    • generation: normalizeQuizAnswer now resolves every answer variant to
      the matching option value (NFKC + whitespace-stripped + letter-prefix
      canonical matching against option values and labels); unknown answers
      pass through untouched
    • grading: resolveAnswerKeyToValue retro-fixes already-generated courses
      whose stored keys hold content variants, mapping them to option values
      when exactly one option matches canonically; ambiguous keys are left
      alone instead of being silently re-pointed
    • tests: letter/content/full-width/trailing-space/multi-answer/unknown
      cases + end-to-end grading with a content-variant key
    • chore(generation): bump version to 0.3.2 for answer-key fix

    The repo requires every publishable @openmaic package change to ship with
    a version bump (CI check-package-version-bumps).

    • style(generation): prettier-format package.json after version bump

    • fix(quiz): use resolved answer representation in review renderers; prettier

    • QuestionCard + quiz-view review modes highlight the option canonically
      matching the stored answer key (answerIncludesOption), so persisted
      content-variant keys highlight correctly after the grading fix
    • prettier formatting on touched files

    • test(quiz): guard possibly-undefined answer in multi-choice test

    • fix(quiz): narrow answer-key matching to exact unique alignment; project review UI from the same resolver

    Per review: matching policy narrows to the DSL option-identity contract.

    • Generation: exact value or unique exact label alignment; unknown and
      ambiguous keys stay unresolved (fail closed). Prompt example unified to
      the DSL object-options + value-answer shape.
    • Grading: persisted keys resolve via the same exact alignment; the
      resolver returns the option's actual value when exactly one option
      matches canonically-or-exactly; ambiguous keys stay untouched.
    • Review UI: QuestionCard and both quiz-view review modes highlight via
      the same resolver projection (answerIncludesOption(q, optionValue)).
    • Removed the broader equivalence rules (NFKC, whitespace stripping, case
      folding, wrapped/prefixed-letter probe) — semantic synonyms such as
      正交/垂直 remain uncovered, hence Refs #1326.
    • fix(quiz): resolve answer keys for persisted keys only; align quiz template with the DSL
    • grading: compatibility resolution (label -> value) now applies to the stored
      answer key only. A learner submission is compared as submitted, so an option
      label can no longer be accepted as a different option's value.
    • quiz-content/user.md: replace the string-options/correctAnswer example with
      the DSL shape the system prompt defines ({ label, value } options and
      answer as an array of option VALUES), so generated keys are values.
    • tests: regression that a label submission grades incorrect while the same
      question's value submission grades correct.
    • test(generation): update scene-prompt golden snapshot for the DSL-shaped quiz template

    The golden test pins the rendered user prompt for every scene kind; the
    quiz-content template now emits the DSL shape, so the snapshot follows.

    • fix(quiz): read label-stored correct answers in the editor's option rows

    quiz-edit-ops.toRows decided correctness with raw value equality, so an
    existing label-keyed quiz showed A/B as correct (the review UI and the grader
    both resolve labels) while every row read as incorrect internally — and the
    next option edit or reorder rebuilt answer from those rows and silently
    dropped the key.

    • toRows now uses the same resolved projection (answerIncludesOption).
    • normalizeQuizAnswer's docstring now describes the exact-only behavior it
      has had since the review: formatting variants are not normalized and
      ambiguous entries pass through untouched.
    • Regressions: an unrelated option-label edit and an option reorder both keep
      a label-stored answer, and a multiple-choice toggle preserves it. All three
      fail against the previous raw comparison.

    Co-authored-by: CrazyPandaP CrazyPandaP@users.noreply.github.com
    Co-authored-by: wyuc wang-yc24@mails.tsinghua.edu.cn

    下载附件