github-featured / nrwl--nx

发布

0 发布 1,252 Git 标签

RSS 订阅源
  • 23.0.0-beta.5 c4ca481574
    比较

    feat(gradle): stream batch task results to nx as they finish (#35487)

    frostbyte_neo 发布于 2026-05-01 21:09:04 +00:00

    Current Behavior

    The Gradle batch executor (@nx/gradle:gradle in batch mode) returns a
    Promise<BatchResults>. The Kotlin batch runner serializes the entire
    result map to a single JSON blob and printlns it once at the end of
    the run. The Node-side executor accumulates stdout chunks and
    JSON.parses them after the JVM exits, so Nx only learns about task
    outcomes in one burst when the whole batch is done.

    The Maven batch executor was migrated to streaming a while ago — it
    returns an AsyncGenerator and the Kotlin runner emits
    NX_RESULT:{json} lines as each task finishes — but the Gradle batch
    executor was never updated to match.

    Expected Behavior

    The Gradle batch executor now mirrors the Maven batch executor's
    streaming protocol, with per-task results streamed live during both
    build and test task execution.

    Kotlin runner (packages/gradle/batch-runner)

    • ResultEmitter writes NX_RESULT:{json} lines to stdout, one per
      task, with a thread-safe dedupe set so emission can happen from
      build/test listeners without double-reporting.
    • runBuildLauncher emits per build task as TaskOutputCapture
      detects the next task's > Task :foo:bar header, with an end-of-build
      flush for the final task.
    • runTestLauncher emits per Nx test task at its class-level
      TestFinishEvent, with a TaskFinishEvent fallback for tasks that
      never produced a class event (compile failure, exclusion). Method-level
      failures are sticky so a later passing method in the same class can't
      mask an earlier failure.
    • NxBatchRunner.main ends with exitProcess(0) so lingering
      non-daemon threads from the Gradle Tooling API can't keep the JVM alive
      after task work completes. The trailing results.forEach { emit } loop
      now only covers finalizeTaskResults-synthesized entries
      (excluded/skipped tasks).

    Node executor

    (packages/gradle/src/executors/gradle/gradle-batch.impl.ts)

    • gradleBatch is now an async function* returning AsyncGenerator<{ task; result: TaskResult }>.
    • streamTasksInBatch spawns the JVM, reads stdout via readline, and
      drains NX_RESULT lines into an in-memory queue, yielding from the
      queue. Yielding inside the readline loop creates back-pressure — slow
      consumers block yield, readline pauses, the OS pipe between Java and
      Node fills, and Java's println blocks on a full pipe. The queue
      decouples reading from yielding so back-pressure can no longer deadlock
      the JVM.
    • Stderr stays inherited so Gradle/JUnit progress flows to the terminal
      in real time.
    • Tasks the runner never reports get yielded as failed at the end so Nx
      never hangs.

    Project graph dependency

    packages/gradle/project.json adds :gradle-batch-runner to
    implicitDependencies. The gradle package bundles the batch-runner JAR
    and references it at runtime via batchRunnerPath; without this, nx affected wouldn't pick up gradle when only the runner changed.

    Bug along the way: className format mismatch

    RegexTestParser.kt records testClassName as the simple class
    name (e.g. MyTest), but Gradle's
    JvmTestOperationDescriptor.className is the fully qualified name
    (com.example.MyTest). The exact-match lookup in the test listener was
    failing for every class, so per-class TestStartEvent/TestFinishEvent
    never matched an Nx task — every Nx task fell through to the
    TaskFinishEvent fallback at the end of the Gradle test task, all
    sharing the same emission time and the cumulative shared output buffer.
    resolveNxTaskId now looks up by FQN first, then by the suffix after
    the last ., so events match either format.

    Known trade-off

    Tests under the same Gradle test task share the captured per-Gradle-task
    output for terminalOutput. With JUnit --parallel the bytes
    interleave anyway, and Gradle's TestLauncher doesn't expose per-test
    stdout segmentation through setStandardOutput — getting truly
    per-class terminalOutput would require either subscribing to
    OperationType.TEST_OUTPUT (which diverts stdout away from the standard
    output stream and didn't reliably fire for some setups in testing) or
    recording fully-qualified class names in the project-graph plugin so we
    can match TestOutputEvent parents precisely. Filed as a follow-up.

    The on-the-wire change matches the existing Maven contract
    (run-batch.ts already special-cases isAsyncIterator), so no Nx core
    changes are needed.

    Related Issue(s)

    下载附件