Commit Graph

15 Commits

Author SHA1 Message Date
Andrey Kumanyaev 0b69295d24 Index composer.json so PHP repos have a dependency graph
The manifest table wired nine formats and composer.json was not one, so a
PHP repo produced no module nodes and no dependency edges at all — only the
generic JSON extractor's top-level key variables, which reach neither
`require` nor `autoload`.

ParseComposerJSON and ParseComposerLock follow the package.json template:
require-dev is Indirect with the block tagged on Replace, versions stay
verbatim because composer.lock carries the resolved ones, and a malformed
manifest yields nothing rather than failing the index. Platform
requirements are skipped — `php`, `hhvm`, `ext-*`, `lib-*` and the
`composer-*` runtime pseudo-packages name no installable package, so a
module node for them could never be depended on by anything.

The manifest node is tagged "json", not "php": it shares its ID with the
JSON extractor's file node, and a php-tagged file would vouch for PHP
presence in a repo holding no PHP source. Module nodes do carry php via
ecosystemLanguage, which the per-repo language census ignores for
KindModule.

composer.json's autoload map is the only statement a PHP repo makes about
which namespaces are its own, so the PSR-4 and PSR-0 prefixes — from both
`autoload` and `autoload-dev`, and accepting composer's string-or-array
value — ride the manifest node. Attributing a VENDOR import to the package
that provides it is not attempted: composer.json says which packages are
required, not which namespace each one owns, and the vendor/pkg to
Vendor\Pkg correspondence is convention rather than rule (monolog/monolog
serves the Monolog namespace). Guessing it would mint wrong edges.

Verified on monolog: 19 composer modules (1 production, 18 dev) with the
autoload root `Monolog => src/Monolog, tests/Monolog`.
2026-07-28 08:47:00 +02:00
Andrey Kumanyaev 8480927952 perf(semantic): gate heavy loads on module probes and defer applies behind the resolve phase 2026-07-18 19:11:19 +02:00
Andrey Kumanyaev b0918503f7 refactor(indexer): drive the full pipeline through graph.Store
Closes the gap between "we extracted a Store interface" and "the
indexer actually uses it". Previously the Store interface existed
(8221a40) and three backends implemented it, but every consumer of
the graph — indexer.New, resolver.New, NewCrossRepo, the temporal /
gRPC / external resolver passes, the contracts bind/wrapper passes,
the modules import linker, the semantic enricher — still typed its
parameter as *graph.Graph. That made the disk backends unreachable
from production code paths and reduced the cross-backend benchmark to
"how fast can we migrate one in-memory graph into another store"
instead of "how fast does the real indexer run with this backend".

This commit rewrites the affected signatures in place:

  *graph.Graph  →  graph.Store

across the indexer, resolver, contracts, modules, semantic, and
related packages. No call sites change behaviour — *graph.Graph
already satisfies graph.Store (via the compile-time assertion in
store.go), so existing callers that hand in a *graph.Graph keep
working unchanged. Disk and remote backends are now also legal
arguments everywhere a graph used to flow.

One small interface change: ResolveMutex() is now a Store method.
The resolver's cross-package coordination (cross-repo, temporal,
external, edge-mutation passes) needs the same serialisation
regardless of backend, so the in-memory-specific carve-out from
the original interface no longer makes sense. Memory store keeps
its existing graph-wide resolveMu; bbolt and sqlite each grew a
dedicated resolveMu separate from their internal write mutexes —
the two protect different invariants and shouldn't share a lock.

What works now that didn't before:

  - indexer.New(boltStore, …)   — full indexing pipeline through bbolt
  - indexer.New(sqliteStore, …) — full indexing pipeline through sqlite
  - resolver.New(anyStore)      — resolver works against any backend
  - All downstream passes (contracts, semantic, modules, clones,
    test-edge, search-index build) take the Store interface

Conformance: all 3 backends still pass the 93-subtest storetest
suite. The 1 166 tests across indexer / resolver / contracts /
semantic / modules / storetest / store_bolt / store_sqlite pass with
the new signatures. go vet ./... clean.

Follow-up commit (bench/store-bench rewrite) will replace the
"migrate in-memory graph into store" pattern with "drive the full
indexer per backend" to produce the apples-to-apples comparison the
old harness only approximated.
2026-05-24 10:19:45 +02:00
Andrey Kumanyaev 3d851104b1 modules, indexer, resolver, graph: model package-manager workspace membership
Detect package-manager workspace roots — an npm/yarn package.json with
a "workspaces" array, a pnpm-workspace.yaml with a packages list, or a
Cargo.toml with a [workspace] members list — and resolve their member
packages, expanding glob patterns against the filesystem.

Materialize the relation in the graph: a synthetic workspace-root node
plus one root-to-member edge per resolved member, under a dedicated
package_workspace_member edge kind, emitted at index time.

When import resolution finds several same-named candidates in different
packages of one repo, prefer the candidate that shares the importing
file's workspace member. The resolver gains a workspace-membership
lookup, fed from the manifests by the indexer, applied as a tie-break
before the existing first-hit fallback so non-workspace repos are
unaffected.
2026-05-21 21:03:46 +02:00
Andrey Kumanyaev a4fb694018 indexer, resolver: resolve npm-alias package imports to local packages
A package.json dependency can be declared as an npm alias —
`"shared": "npm:@acme/shared-lib@1.4.0"` — so an `import x from
'shared'` actually refers to `@acme/shared-lib`. Until now the bare
specifier was treated as an external dependency, so a cross-package
edge to a locally-vendored `@acme/shared-lib` was dropped.

The package.json scanner now captures the real package name on the
parsed Spec. A new resolve-time hook walks the importing file's
nearest-ancestor package.json, and when the import specifier matches
an npm-alias dependency key it rewrites the specifier to the real
package name before lookup — handling scoped, plain, and
no-version alias forms, both dependencies and devDependencies, and
sub-path imports. Both the per-repo and cross-repo import resolvers
apply the rewrite; a sub-path import falls back to the package node.
When the real package is not locally indexed the import resolves to
an external stub exactly as before.
2026-05-21 18:36:17 +02:00
Andrey Kumanyaev cd27185b86 daemon, indexer: parallelise warmup + collapse O(R·E) passes
Cold-start warmup on a 490-repo workspace was taking 13+ minutes (and
hung on warmups where the snapshot failed to write).

Five compounding issues, all fixed here:

- warmupDaemonState looped sequentially; replaced with a bounded
  worker pool (min(NumCPU, 12)) gated by BeginParallelBatch.

- TrackRepoCtx / ReconcileRepoCtx called ReconcileContractEdges per
  repo, each walking every edge in the shared graph to evict stale
  EdgeMatches and rebuilding the matcher across every indexer. Skip
  under deferGlobalPasses; RunGlobalResolve fires the single final
  reconcile after the loop.

- RunDeferredPasses called idx.resolver.ResolveAll per repo. With R
  repos and E edges in the shared graph this is O(R·E) — 95 % of
  warmup CPU on a 490-repo / 1.9M-edge graph. Added
  skipResolveInDeferred; RunDeferredPassesAll runs one global
  resolver.New(graph).ResolveAll at the end.

- modules.LinkImports walked g.AllNodes() (963K nodes) once per repo
  per manifest type. New LinkImportsIn(g, importNodes, ...) takes a
  caller-supplied node slice; extractOneModuleManifest feeds it the
  repo's own KindImport nodes from GetRepoNodes.

- daemon_snapshot init registered []map[string]string but not
  map[string]string, so every snapshot write aborted with
  "gob: type not registered for interface". No snapshot ever
  persisted, forcing full re-track on every restart. Register the
  missing type.

Also adds per-phase timing logs (parallel_parse / deferred_passes_all
/ global_resolve / end_batch) and per-repo elapsed warnings above 2s
so future regressions are visible in daemon.log.

Measured on a 490-repo / 1.9M-edge / 963K-node workspace: warmup
goes from 13+ min (full track, no snapshot) to ~51s on the reconcile
path (parallel_parse 30s, deferred 0.4s, global_resolve 17s,
end_batch 3s).
2026-05-13 00:35:58 +02:00
Andrey Kumanyaev 2a8729b3df mcp, agents, hooks: surface remaining graph edges and refresh installed instructions
Add seven new `analyze` kinds — channel_ops, goroutine_spawns,
  field_writers, annotation_users, config_readers, event_emitters,
  error_surface — so the EdgeSends/Recvs/Spawns/Reads/Writes/Annotated/
  ReadsConfig/Emits/Throws edges emitted by recent parser work are
  reachable through the graph-summary API. Each handler ships with a
  typed row, GCX encoder, and unit tests.

  Refresh every surface that `gortex install` writes to user agents:
  InstructionsBody and GlobalInstructionsBody, all five Claude Code
  slash commands, both global skill bodies, the PreToolUse soft-deny
  guidance, and the post-task briefing now mention every supported
  analyze kind, every node kind, and the gortex enrich CLI workflow.
  The repo's own CLAUDE.md is updated to mirror the new template.

  Incidental lint cleanup: wrap unchecked Fprintf in cmd/gortex/export.go
  and internal/semantic/lsp/provider_test.go, drop dead `-1` initialisers
  in internal/modules/scanner.go, and tighten propPair construction in
  internal/exporter/cypher.go.
2026-05-03 13:17:17 +02:00
Andrey Kumanyaev 4133b96521 modules, indexer: parse yarn.lock and pnpm-lock.yaml for resolved npm versions
After package.json + package-lock.json shipped npm ecosystem
  coverage for the most common toolchain, projects using yarn or
  pnpm got nothing — their lockfiles carry the resolved versions
  but the dispatch loop didn't know about them. This adds parsers
  for both formats so any of the four major npm-ecosystem
  manifest shapes lands in the same canonical
  module::npm:<name>@<version> identity.

  - modules: ParseYarnLock walks yarn-classic's pseudo-yaml
    line-oriented format. Block headers are unindented lines
    ending in `:` containing one or more `<name>@<range>` entries
    joined by `, `. parseYarnHeaderNames extracts the package
    names — scoped packages (`@types/node@^20`) need the version
    separator search to skip past the leading scope `@`, so the
    helper checks the prefix and offsets the search accordingly.
    Multi-range blocks (`lodash@^4.0.0, lodash@^4.17.0`) dedupe
    to one Spec via the in-block name set. Berry / yarn-2+
    lockfiles use real YAML and a different shape; recognized as
    a future dispatch row when needed.

  - modules: ParsePnpmLock walks pnpm v6's YAML structure via
    line-oriented scan rather than a full YAML parse. Per-pkg
    keys are `/<name>@<version>` at exactly two-space indent in
    the `packages:` section; a top-level unindented key ends the
    section. Strips peer-dep suffixes encoded in the key
    (`some-pkg@1.0.0_react@18.2.0` → version 1.0.0) so canonical
    versions stay clean. Scoped packages use LastIndex of `@`
    rather than first-index because the scope itself starts with
    `@`.

  - indexer: yarn.lock and pnpm-lock.yaml join the manifest
    dispatch table next to package-lock.json. Both share the
    nil ownPathFromSrc — like package-lock, lockfiles share
    their identity with package.json's name field. manifestLanguage
    extends to recognise yarn.lock (yarn tag) and pnpm-lock.yaml
    (yaml tag).

  - tests: 6 new cases — yarn-classic basic + multi-range dedup +
    empty input, pnpm v6 basic + peer-suffix strip + empty +
    no-packages-section.  End-to-end on a synthetic fixture with
    one yarn.lock + one pnpm-lock.yaml produces three module
    nodes (lodash, react, vue) with their expected file→module
    edges. Race-enabled tests pass.

  Manifest dispatch design now validated across six format
  families: TOML (pyproject, Cargo), JSON (package.json,
  package-lock), XML (pom.xml), pseudo-YAML (yarn.lock), real
  YAML (pnpm-lock), and custom (go.mod, requirements.txt).
  Each new ecosystem remains one parser + one dispatch row.
2026-05-03 11:31:38 +02:00
Andrey Kumanyaev 1097421834 modules, indexer: parse package-lock.json for resolved npm versions
Agents asking "is lodash@4 still in use anywhere" got the answer
  in semver-range form when only package.json was parsed —
  `^4.17.0` doesn't tell them which point release actually
  shipped. Adding the lockfile parser closes that gap so the
  graph carries both the declared range and the resolved version
  side-by-side.

  - modules: ParsePackageLockJSON walks the v2/v3 lockfile shape
    (top-level `packages` map keyed by `node_modules/...`). The
    empty-string key — the root project's own entry — is skipped
    same as ParsePackageJSON drops its own `name`. The
    `node_modules/` prefix is stripped from each key; nested
    transitive paths like `foo/node_modules/bar` preserve their
    full chain so duplicate-name entries at different versions
    stay distinguishable as separate module nodes.

  - modules: v1 lockfiles return nil. The shape is fundamentally
    different (top-level `dependencies` map, no `packages`) and
    rare in the modern npm ecosystem — a parallel parser would
    duplicate most of this logic for marginal value. The
    `lockfileVersion` field on the manifest tells future callers
    which shape they have, so the v1 parser can land alongside
    when prioritised.

  - modules: dev/optional classification reuses the Spec.Replace
    field, matching the convention from package.json /
    pyproject.toml / Cargo.toml — production stays empty,
    indirect kinds carry a tag.

  - indexer: adds package-lock.json to the manifest dispatch
    table next to package.json. ownPathFromSrc is nil because
    the lockfile shares its identity with package.json's name
    field — no separate own-name notion. manifestLanguage folds
    package-lock.json in under the json tag.

  - tests: 3 cases — full v3 lockfile with scoped packages and
    nested-transitive paths to pin both edge cases, v1
    rejection (returns nil), and empty-input handling.

  When both manifests are present, both emit independent module
  nodes — a package with a semver range (`^4.17.0`) and a
  resolved version (`4.17.21`) produces two distinct nodes. The
  TS import edge resolves to both via longest-prefix match —
  noisy but truthful. Agents querying "is lodash@4 still in use"
  should filter to lockfile-resolved nodes (numeric versions);
  agents querying "what does package.json declare" should look
  for semver-range nodes. Lockfile-supersedes-manifest dedup is
  a future refinement.
2026-05-03 03:53:10 +02:00
Andrey Kumanyaev df5909fce6 modules, indexer: parse pom.xml so Maven coordinates land in the graph
After Go, npm, PyPI, and Cargo, Java is the next ecosystem with
  enough deployed footprint to justify joining the dependency
  graph. Adding it also exercises the dispatch loop with an XML
  format alongside the existing JSON/TOML pair — confirming the
  design isn't accidentally tied to map-shaped manifests.

  - modules: ParsePomXML walks <dependencies><dependency> entries
    via encoding/xml. Maven coordinates take the form
    `groupId:artifactId`, so the whole pair becomes Spec.Path —
    keeping the Path field a single string lets the longest-prefix
    matcher in LinkImports work without a Maven-specific code
    path. Versions live as a separate child element. Scope handling
    mirrors the npm/pyproject convention: missing or `compile`
    scope is production (no Replace tag, Indirect=false); test /
    provided / runtime / system / import each stamp Replace =
    "<scope>" and Indirect = true so cleanup queries can scope to
    production-only deps uniformly across ecosystems.

  - modules: property substitution (`${spring.version}`,
    `${project.version}`) is left verbatim. Resolving it requires
    walking <properties> and parent-pom inheritance, which the v1
    deliberately defers — the canonical source-of-truth for
    resolved versions in Maven is `mvn dependency:tree` output,
    not the raw pom, so a future enhancement can either parse
    properties locally or consume the resolved-tree dump from
    Maven itself. The verbatim form at least keeps the dependency
    visible as a node.

  - modules: alphabetical sort within the parser even though XML
    preserves source order — diff-able output across runs matches
    the convention from the other parsers.

  - indexer: extractExternalModules's dispatch table grows another
    row. readPomXMLOwnName builds the project's own
    `groupId:artifactId` coordinate so LinkImports filters Java
    workspace self-references — a parent project shouldn't
    accidentally resolve a child module to an external dep with
    the same coordinate. manifestLanguage extends to fold pom.xml
    in under the xml tag.

  - tests: 3 cases — full block with all four scope variants
    (missing + compile both production, test + provided indirect
    with scope tag), incomplete-coordinate skipping (deps with
    only groupId or only artifactId are dropped), and the
    property-version verbatim contract pinned. End-to-end on a
    synthetic fixture with Spring + JUnit produces two correctly
    classified module nodes.
2026-05-03 02:49:07 +02:00
Andrey Kumanyaev 08f01f88fa modules, indexer: parse Cargo.toml so Rust crates land in the graph
After Go, npm, and PyPI shipped, Rust was the next ecosystem
  blocking dependency-graph queries on real codebases. Adding
  Cargo to the manifest-dispatch loop costs one parser plus one
  table row, which is exactly the design point the dispatch loop
  was meant to validate — each new ecosystem stops looking like a
  feature and starts looking like a config entry.

  - modules: ParseCargoToml walks [dependencies], [dev-dependencies]
    (canonical) plus [dev_dependencies] (legacy underscore form
    occasionally seen in older manifests), and [build-dependencies].
    cargoBlock destructures each entry's value as either a bare
    version string ("1.0", "^1.0") or a table form
    (`tokio = { version = "1.30", features = [...] }`). Both fold
    through one Spec-construction path so adding more table-keyed
    fields (`registry = "..."`, `package = "..."`) is a trivial
    extension.

  - modules: path-only and git-only deps without a version field
    are deliberately skipped. `local = { path = "../local" }` and
    `repo = { git = "..." }` resolve to workspace-internal or
    revision-pinned sources, not versioned crates.io releases. The
    graph models external versioned identity, and emitting nodes
    for these would muddy "what does this depend on" queries
    without contributing meaningful version info.

  - modules: dev/build kinds repurpose Replace as a kind tag, the
    same idiom npm and pyproject use. Cleanup queries scoping to
    "production-only deps" stay uniform across all four
    ecosystems.

  - indexer: extractExternalModules's dispatch table grows another
    row. readCargoTomlOwnName mirrors the per-ecosystem own-name
    helpers — workspace-internal crate references won't
    accidentally match external crates of the same name at
    LinkImports's longest-prefix scan. manifestLanguage extends to
    fold Cargo.toml in alongside pyproject.toml under the toml
    tag.

  - tests: 3 cases — all four blocks parsed with the path/git
    skip exercised, underscore dev_dependencies accepted as the
    canonical dev kind, and empty / no-deps manifests handled
    gracefully. End-to-end on a synthetic Rust fixture produces
    three module nodes (serde, tokio, assert_cmd) with Cargo.toml
    linking to each via EdgeDependsOnModule.

  Per-source import→module edges for .rs files are gated on the
  Rust extractor emitting KindImport nodes, which it doesn't yet
  do. The file-level relationship is in place today; per-file
  edges land transparently when import emission gets added to
  the Rust extractor.
2026-05-03 02:38:28 +02:00
Andrey Kumanyaev 8ec96ea3e9 modules, indexer: parse pyproject.toml and requirements.txt for PyPI deps
Gortex's own eval/ subdirectory had a pyproject.toml that was
  invisible to every dependency-graph query, and any Python repo
  agents indexed got the same blank result. Adding PyPI to the
  manifest-dispatch loop makes Python imports resolve to versioned
  module nodes the same way Go and npm imports do — and exercises
  the dispatch design that the Go+npm pair previously couldn't
  prove out by itself.

  - modules: ParsePyProject handles both PEP 621 and Poetry shapes
    in one pass. PEP 621 dependencies = ["pkg>=1.0", ...] go
    through splitPEP508 to drop environment markers, URL installs,
    and extras suffixes. Optional groups (project.optional-
    dependencies) reuse the Spec.Replace field as a group tag —
    same idiom as npm's dev/peer/optional, so cleanup queries can
    scope by group without growing Spec for each ecosystem's
    classification axis. Poetry tables (`requests = { version =
    "^2.0", extras = [...] }`) destructure correctly to extract
    just the version. The python = "^3.10" interpreter constraint
    is deliberately filtered — it's not a dependency, and emitting
    a "python" module node would mislead every downstream query.

  - modules: splitPEP508 takes a PEP 508 requirement string into
    (name, version). Environment markers (`; python_version<'3.9'`),
    URL installs (`pkg @ https://…`), and extras (`flask[async]`)
    are all dropped at parse time — they encode install context,
    not dependency identity. The graph cares about the latter.

  - modules: ParseRequirementsTxt walks pip-style files line by
    line, skipping comments, blank lines, and install-mode
    directives (`-r`, `-e`, `--index-url`). Each surviving line
    goes through splitPEP508. Recursive `-r` includes are
    deferred — first-order coverage is the 90th-percentile case
    and pulling in transitive files raises questions about loop
    detection that the v1 doesn't need to answer.

  - indexer: extractExternalModules's dispatch table grows two
    new rows. pyproject.toml gets readPyProjectOwnName for own-
    module filtering (mirrors readPackageJSONOwnName); the
    [project] name field carries the package's own identity.
    requirements.txt has no own-name notion so the
    ownPathFromSrc field is nil — LinkImports already treats nil
    as "no own-module filter", so the wiring lands without a
    special case.

  - indexer: manifestLanguage extended for toml + text so the
    synthetic file nodes surface correctly in Brief listings.

  - tests: 4 cases — PEP 621 deps with extras and an optional
    group, Poetry mixed string/table syntax with the python-
    interpreter filter, requirements.txt with comments and
    install-mode directives, and splitPEP508 itself across the
    major shapes (constraint, extras, environment marker, URL).
    Real-repo re-index of Gortex's eval/ produces 8 KindModule
    nodes, 8 file→module edges from pyproject.toml, and many
    import→module edges from .py source files — pytest, hypothesis,
    docker, jinja2, datasets, tabulate all correctly versioned.
2026-05-03 02:26:58 +02:00
Andrey Kumanyaev f1b5e6cf9d modules, indexer: parse package.json so npm deps land in the graph
After Go modules shipped, an agent indexing a TypeScript or
  full-stack repo got nothing from the dependency-graph axis —
  package.json was opaque, and TS import nodes never resolved to
  versioned module nodes. This adds a parallel npm pass alongside
  the Go one, with a manifest-dispatch loop in the indexer so
  future ecosystems (Cargo.toml, pyproject.toml, pom.xml) land as
  single new rows rather than as forks of extractExternalModules.

  - modules: ParsePackageJSON unmarshal walks all four dependency
    blocks (dependencies, devDependencies, peerDependencies,
    optionalDependencies). Production deps get empty Replace;
    dev/peer/optional repurpose the Replace field as a kind tag
    so agents can scope to "production-only" deps without growing
    the Spec struct for an ecosystem-specific axis. Each block is
    sorted alphabetically before append so test output is
    deterministic — npm libraries reasonably expect stable
    iteration even though Go's JSON map order is randomised.

  - modules: version strings stay verbatim. npm semver ranges
    (`^1.2.0`, `~3.4.1`, `>=2 <3`) aren't normalised — resolved
    versions belong in the lockfile, not package.json. A future
    package-lock / pnpm-lock extractor will supersede the manifest
    version with the resolved one.

  - indexer: extractExternalModules is restructured around a
    manifest-dispatch table. Each row pairs a relative path with
    a parser and an own-name extractor; extractOneModuleManifest
    runs the per-manifest pipeline (read, parse, build artifacts,
    emit synthetic file node, run LinkImports). Adding the next
    ecosystem is a one-line entry plus a parse function on
    modules. The synthetic file node's language tag is now picked
    by manifestLanguage rather than hardcoded to "go" — package.json
    surfaces as language=json in Brief listings.

  - indexer: readPackageJSONOwnName extracts the manifest's `name`
    field — the npm equivalent of go.mod's `module` directive —
    to feed LinkImports's own-module filter. Workspace setups
    where a package imports its own name (`import "@my-app/util"`)
    won't accidentally collide with external deps at the longest-
    prefix scan.

  - tests: 4 cases — all four dependency blocks parsed correctly
    with the right indirect/kind classification, empty + malformed
    input handled gracefully, and the stable-alphabetical-sort
    contract pinned. The existing LinkImports tests already cover
    the cross-ecosystem resolution path because LinkImports is
    ecosystem-agnostic — TS imports stamp meta.path the same way
    Go imports do, and the longest-prefix matcher works on the
    string regardless of source language. End-to-end on a
    synthetic fixture (one package.json + one main.ts importing
    react and lodash) produces three module nodes, three
    file→module edges, and two import→module edges with the
    versions correctly attached including semver ranges.
2026-05-03 02:17:16 +02:00
Andrey Kumanyaev b1e3015229 modules, indexer: link Go imports to their resolved module nodes
After the manifest pass landed file→module edges, an agent asking
  "which files use cobra" still had to walk through file nodes
  rather than join directly through the import nodes the parser
  already emits. Each Go import was a graph node with a path string
  but no edge to the version-specific module — agents reaching for
  "show me everything depending on lodash@4" had to text-match. This
  closes the gap so import nodes become first-class consumers in
  the dependency graph.

  - modules: LinkImports walks every KindImport node, reads its
    path from meta, and finds the longest-prefix match against the
    spec list. Specs are pre-sorted by descending path length so
    the prefix scan picks the most specific match in one pass —
    important for the `github.com/foo/bar` vs `github.com/foo/bar/v2`
    disambiguation that Go's major-version semantics rely on.
    Versioned imports correctly resolve to the longer spec, so an
    `import github.com/foo/bar/v2/sub` lands on the v2 module node
    even when both v1 and v2 are present in go.mod.

  - modules: repo-internal imports (the indexed module's own path)
    are filtered inside LinkImports rather than at the call site —
    keeping the policy with the matcher means future call sites
    (other manifest formats, multi-module monorepos) can't
    accidentally drop the filter. Stdlib imports (fmt, context,
    errors) match no external spec and produce zero edges, which
    is the correct behaviour: they aren't tracked dependencies.

  - indexer: extractExternalModules invokes LinkImports after the
    file→module emission, with the own-module path read from
    go.mod's `module` directive via a local readGoModModulePath
    helper. The duplication with coverage.ReadModulePath is
    deliberate — keeping both inline avoids a cross-package
    dependency that would couple the indexer to the coverage
    package for one trivial parse. Both are small enough that the
    duplication is cheaper than the layering compromise.

  - tests: 3 cases — longest-prefix match exercising exact match
    + sub-package match + own-module skip simultaneously, versioned-
    import disambiguation that pins v2 over v1 when both specs
    are declared, and stdlib-import-skipped which verifies
    no-match produces zero edges. Real-repo re-index of the
    Gortex tree climbs depends_on_module edge count from 122
    (file→module only) to 281 — the new ~159 import→module edges
    correctly resolve to specific versions like cobra@v1.10.2,
    yaml.v3@v3.0.1, mcp-go@v0.49.0.
2026-05-03 02:09:24 +02:00
Andrey Kumanyaev 0688256559 modules, indexer: model external Go dependencies as graph nodes
An agent asking "what external packages does this repo depend on",
  "are we still on lodash@4 anywhere", or "which deps have an indirect
  flag we can drop" had no graph answer — go.mod was opaque to every
  query tool. This adds a one-shot manifest pass that turns each
  require directive into a queryable module node with full version
  metadata.

  - modules: new package. ParseGoMod walks the manifest in a single
    line-oriented pass, handling single-line require, grouped
    `require ( ... )` blocks, `// indirect` markers, and replace
    directives in both single-line and block forms. Replacement
    targets stamp onto the matching require's Replace field after
    the scan completes, so block-form replaces declared after the
    requires still attach correctly. BuildGraphArtifacts emits one
    KindModule node per (ecosystem, path, version) tuple — de-duped
    via a seen-set so a malformed manifest with duplicates still
    produces one node — plus one EdgeDependsOnModule per spec.
    ModuleNodeID follows the canonical `module::<ecosystem>:<name>@<version>`
    convention; shortName extracts a friendly Brief label and
    correctly peels Go's `/vN` major-version suffix so
    `github.com/foo/bar/v2` reads as `bar`.

  - indexer: extractExternalModules is a one-shot manifest pass
    modeled on the existing extractGoModContracts. go.mod has no
    registered language extractor, so it never reaches the per-file
    pipeline; the pass reads `<rootPath>/go.mod` directly, parses
    it, and writes the resulting nodes plus a synthetic KindFile
    node for go.mod itself into the graph. The synthetic node is
    important — without it, the EdgeDependsOnModule edges would
    dangle from a missing source endpoint after applyRepoPrefix
    runs in multi-repo mode. Wired into both the bulk IndexCtx
    finishing block and the deferred-resolve path that the multi-
    repo orchestrator uses, alongside the existing contract passes.

  - tests: 7 cases covering single-line + block require, indirect
    markers, replace directives, empty-input edge cases, ID format,
    deduplication during artifact construction, and shortName
    behaviour for plain paths and vN suffixes. Real-repo re-index
    of Gortex's own go.mod produces 122 module nodes with full
    meta (ecosystem / path / version / indirect / replace) plus
    122 file→module edges.

  Per-import edges from each Go import node to the matching module
  node, multi-ecosystem support (package.json, pnpm-lock,
  requirements.txt, Cargo.toml, pom.xml), and incremental re-index
  on go.mod changes are tracked as v2 follow-ups — the v1 file-
  level edge is already enough for the dependency-listing queries
  that motivated this work.
2026-05-03 00:57:17 +02:00