github-featured / nrwl--nx

发布

0 发布 1,252 Git 标签

RSS 订阅源
  • 22.1.0-rc.3 b3e6b03d82
    比较

    chore(core): restructure metrics collector for performance and maintainability (#33483)

    frostbyte_neo 发布于 2025-11-14 16:56:51 +00:00

    Current Behavior

    The metrics collector has several inefficiencies and architectural
    issues:

    • Complex hierarchical data structure (ProcessTreeMetrics) that
      doesn't align with how the data is consumed
    • Separate MetadataStore struct with unnecessary indirection
    • Full metadata resent to all subscribers on every collection cycle
    • Mutable parameters passed through collection functions instead of
      functional return values
    • Repeated allocations and clones across collection cycles
    • CollectionRunner tightly coupled to NAPI, making it untestable in
      pure Rust

    Expected Behavior

    This PR restructures the metrics collector for better performance,
    testability, and maintainability:

    Architectural Changes

    1. Flat Process Model: Replaced hierarchical ProcessTreeMetrics
      with a flat Vec<ProcessMetrics>, simplifying data flow

    2. Group-Based Organization: Introduced GroupInfo and GroupType
      to logically organize processes:

      • MainCLI - Nx CLI process and its subprocesses
      • Daemon - Nx daemon and its children
      • Task - Individual task execution processes
      • Batch - Batch execution with multiple tasks

    3. Incremental Metadata Updates:

    • Track which groups and processes have been sent using
      Arc<DashMap<String, GroupInfo>> and Arc<DashMap<String, ProcessMetadata>>
    • Only send new metadata to subscribers instead of full state every
      cycle
    • New subscribers receive full metadata on first update via
      needs_full_metadata flag
      • Automatic cleanup of dead process/group metadata
    1. Shared State with Arc:
    • Metadata maps shared between ProcessMetricsCollector and
      CollectionRunner using Arc<DashMap>
      • Eliminated duplicate metadata storage
      • Single source of truth for all metadata
    1. Functional Programming Pattern:
    • Collection functions now return Result<MetricsCollectionResult>
      instead of mutating parameters
      • Cleaner error handling with inspect_err and map
      • Easier to reason about data flow
      • Removed ~100 lines of code by consolidating logic
    1. Channel-Based Communication:
      • Decoupled CollectionRunner from NAPI using crossbeam_channel
      • Collection thread sends metrics via channel to listener thread
      • Listener thread receives metrics and notifies NAPI subscribers
      • CollectionRunner is now NAPI-free and fully testable in pure Rust
    • Non-blocking collection (subscriber callbacks don't block metrics
      collection)

    Performance Optimizations

    • Pre-allocated Vec capacity when combining metrics from different
      sources
    • Eliminated unnecessary HashMap clones during metadata updates
    • Single-pass insertion into DashMap during string key conversion
    • Reduced memory allocations in hot paths
    • Collection thread never blocks on JavaScript callbacks

    Code Quality Improvements

    • Testability: Added 7 pure Rust unit tests for CollectionRunner:
      • Group creation with different registration types
      • Incremental metadata updates
      • Dead group cleanup
      • All tests pass without requiring NAPI/Node.js runtime
    • Clearer separation of concerns between collection and notification
    • Better comments explaining incremental update strategy
    • More idiomatic Rust patterns throughout
    • Updated TypeScript type exports to match new structure

    Threading Model

    Before:

    CollectionRunner (mixed collection + NAPI notification)

    After:

    CollectionRunner (pure Rust, testable)
  └─> Channel
      └─> Listener Thread
          └─> NAPI ThreadsafeFunction
              └─> Subscribers

    Testing

    • ✅ 241 Rust tests passing (including 7 new CollectionRunner tests)
    • ✅ Native module builds successfully
    • ✅ TypeScript types updated and exports verified

    Related Issue(s)

    Part of ongoing metrics collector optimization work.

    下载附件