Commit Graph

6 Commits

Author SHA1 Message Date
Andrey Kumanyaev c25d306baa perf(graph,coverage): move coverage enrichment out of nodes.meta into a typed sidecar
Change A, coverage domain (mirrors the churn sidecar 5d19188). Coverage %
+ stmt counts now persist in a typed coverage_enrichment table
(node_id PK + repo_prefix + coverage_pct/num_stmt/hit) via a new optional
CoverageEnrichmentWriter/Reader capability on both backends, with a
conformance case.

- The coverage enricher writes the sidecar (batched) and, on success,
  strips the Meta stamps + skips the AddBatch so the node blob stays
  lean; on a sidecar write failure it falls back to persisting Meta via
  AddBatch so coverage is never lost.
- All six coverage readers (coverage_gaps, coverage_summary, health_score,
  knowledge_gaps, inspections, replay_episode) now batch-load the sidecar
  once via coverageByID() and read through coveragePctFrom(), which falls
  back to Meta["coverage_pct"] for un-migrated DBs / capability-less
  backends. EdgeCoveredBy edges (and their edge-level coverage_pct) are
  unchanged.

Recompute-on-next-enrich migration; no destructive backfill. Tests:
conformance on both backends, enricher round-trips the sidecar (and
leaves no coverage in node Meta), and the coverage analyzers + the
enrich-coverage MCP path read it.
2026-06-02 09:06:42 +02:00
Andrey Kumanyaev eec5b86243 chore(cleanup): scrub stale ladybug/Kuzu references from comments
The ladybug/Kuzu backend is gone, but doc-comments across the graph,
indexer, analysis, query, and MCP layers still described query plans in
terms of Cypher / liblbug / cgo round-trips and named store_ladybug as
the sole implementor of various Store capabilities. Generalize them to
backend-agnostic wording (SQLite is now the on-disk backend) and delete
the obsolete ladybug-internals trivia. The rationale (why a capability
exists, the row-count tradeoffs) is preserved; only the backend naming
changed.

Two small non-comment follow-ups ride along: normalizeBackendTag's dead
"ladybug" snapshot-tag case becomes "sqlite", and enrich_churn's help
text drops the LadyBug write-lock name (the route-through-the-daemon
rationale still holds for SQLite's single-writer lock). Touched files
were re-run through gofmt (the project's make fmt standard), which
realigned a few struct blocks. No behavior change.

Excludes resolver.go and indexer.go, which also carry unrelated WIP.
2026-06-01 10:24:16 +02:00
Andrey Kumanyaev 758f780884 fix(coverage,blame): persist enriched node Meta on disk backends
Both enrichers stamped node Meta in place — coverage_pct/coverage in
coverage.EnrichGraph, last_authored in blame.EnrichGraph — but never
wrote the symbol node back through the store (blame wrote back only the
person KindTeam node, not the blamed symbol). On the in-memory backend
that persists via pointer identity; on disk backends the stamp is
discarded the moment AllNodes' slice goes out of scope, so
analyze:coverage_gaps / ownership / stale_code and health_score's
coverage + recency axes were silently empty even after a successful
`gortex enrich coverage|blame`.

Collect the stamped nodes and round-trip them via AddBatch, matching
releases/churn which already do this. Verified on the ladybug backend:
blame now persists last_authored on 597/597 nodes (was 0).
2026-05-29 15:17:14 +02:00
Andrey Kumanyaev a3f5101ff9 refactor: mcp.Server.graph + analysis/etc. take graph.Store, not *graph.Graph
Mechanical interface-widening across the codebase so the daemon
can run on different storage backends (memory, ladybug, sqlite,
duckdb). Every public function that previously took
*graph.Graph as a parameter now takes graph.Store (the
interface *graph.Graph already implements).

What changed:

  - internal/mcp/Server.graph: *graph.Graph -> graph.Store
  - 55 files across 18 packages: parameter signatures only
  - 3 struct fields where the parameter-type change cascaded
    (wiki.Inputs.Graph, wiki.Generator.graph, docs.Deps.Graph,
    dataflow.Engine.g, skills.Generator.graph)
  - 2 in-package functions in internal/graph: ClassifyZeroEdge,
    CaveatForZeroEdge

No behavioural change: every method called on a parameter is on
the graph.Store interface, *graph.Graph satisfies graph.Store,
and every existing caller continues to work because Store is
strictly more permissive than *graph.Graph.

What this unlocks: the daemon can now construct a Server with
any graph.Store implementation (store_ladybug, store_sqlite,
store_duckdb), not just the in-memory *graph.Graph. The capability
interfaces (PageRanker, CommunityDetector, ComponentFinder,
KCorer, SymbolSearcher, VectorSearcher) auto-engage via the
existing type assertions in handleAnalyze*. Cmd/gortex/server.go
backend selector flag lands in the next commit.

Driven via 4 parallel agents per leaf package (audit/search,
dataflow/query/exporter, wiki/semantic/contracts/resolver,
releases/blame/cochange/coverage/docs/server/skills/sql) plus
hand-edits for the cross-cutting bits.
2026-05-25 19:39:52 +02:00
Andrey Kumanyaev 24f088e423 blame, coverage, parser, semantic, sql: produce edges and nodes the schema declared but extractors never emitted
A graph-builder audit turned up 22 cases where node and edge
kinds were declared but no producer ever populated them — agents
asking "who authored X", "which tests cover Y", or "what columns
does this query write" got empty results, and several signals
the spec promised cross-language only existed in the Go extractor.

Schema fixes
- blame.EnrichGraph emits KindTeam person nodes (ID
  team::<email>, meta.kind=person, repo-scoped via RepoPrefix)
  plus EdgeAuthored edges alongside the existing meta.last_authored
  stamp.
- coverage.EnrichGraph inverts each EdgeTests pointing at a
  covered subject and emits EdgeCoveredBy carrying
  meta.coverage_pct; 0%-covered subjects are skipped so the
  relation reflects actual coverage rather than test existence.
- parser.ParseTree exposes HasParseErrors / CountParseErrors and
  indexer.stampParseErrors stamps meta.has_parse_errors /
  meta.parse_errors on the KindFile node so index_health can rank
  broken sources.

Go fidelity
- golang.go switches value-side selectors and bare-ident value
  uses from EdgeReferences to EdgeReads (LHS-of-assign already
  emitted EdgeWrites). Return-statement reads are captured so
  the "every value use is a read" rule holds end-to-end.
- go_function_shape.emitGoClosureCaptures walks each func_literal
  for free variables and emits EdgeCaptures with meta.name,
  honoring closure parameters, range-clause loop vars, type-
  switch bindings, and var/const decls as scopes that suppress
  the capture.

Function shape generalized
- New ts_/py_/rust_/java_function_shape.go helpers emit
  KindParam, EdgeParamOf, EdgeTypedAs, EdgeReturns,
  KindGenericParam + EdgeMemberOf for TypeScript, Python, Rust,
  and Java. Each ships a type-canonicalizer that strips idiomatic
  wrappers (Promise/Optional/Result/Mono/List/Box/Vec/Awaited/
  PEP-604 unions, etc.) and skips primitives so the emitted
  edges land on real type nodes.

Async spawns
- TS: await_expression and Promise.all/allSettled/race/then.
- Python: await and asyncio.{gather, create_task,
  ensure_future, run, wait, wait_for, shield}.
- Rust: await_expression and tokio::spawn / spawn_blocking,
  async_std::task::spawn, smol::spawn.
- Kotlin: launch / async / runBlocking / withContext /
  coroutineScope + .await(). Lambdas are walked because the
  Kotlin extractor doesn't materialise them as graph nodes.
- C#: await_expression and Task.Run / Task.Factory.StartNew,
  ThreadPool.QueueUserWorkItem, Parallel.{ForEach, For, Invoke}.

LSP overhaul
- protocol.go gains CallHierarchy / TypeHierarchy capabilities
  and item types; provider.go wires prepareCallHierarchy +
  outgoingCalls / incomingCalls and prepareTypeHierarchy +
  supertypes / subtypes.
- enrichCallHierarchy promotes text_matched / ast_inferred call
  edges to lsp_resolved and adds calls the AST extractor missed
  (typically cross-file).
- enrichTypeHierarchy emits EdgeExtends / EdgeImplements for
  every indexed type or interface — the biggest non-Go win
  because AST extraction can't follow `extends X` / `implements I`
  across files.
- Hover / references / implementation resolve to the actual
  identifier column instead of col=0. The old default empty-
  resulted every method declaration in indented contexts: jdtls,
  omnisharp, kotlin-language-server, and rust-analyzer all
  require the position to land on the identifier itself.

Column-level SQL
- sql.ExtractColumns and sql.ColumnNodeID extract column refs
  from INSERT col-lists, UPDATE SET assignments, and single-
  table SELECT projections. Multi-table SELECTs (any JOIN)
  suppress column emission because v1 can't disambiguate which
  table each column belongs to without a real SQL parser.
- go_sql.go threads ColumnRef alongside TableRef and emits
  KindColumn nodes plus EdgeReadsCol / EdgeWritesCol so
  "which functions read this column" works alongside the
  table-level EdgeQueries.
2026-05-09 02:22:00 +02:00
Andrey Kumanyaev 4fb206ced9 coverage, mcp: stamp per-symbol coverage_pct from Go cover profiles
An agent asking "which functions in pkg/auth aren't tested",
  "what's the coverage of HandleLogin", or "find untested code in
  files I just changed" got binary answers from the existing
  get_untested_symbols tool — a symbol either had a test edge or
  it didn't. That heuristic misses partially-covered code and
  ignores branch-level signal that a real cover profile carries.
  This adds a one-shot enrichment that projects Go cover.out
  segments onto function nodes so coverage becomes a numeric
  property the graph carries directly.

  - coverage: new package. Parse walks the cover-profile format
    (one segment per line: <file>:<sl>.<sc>,<el>.<ec> <numStmt>
    <count>), skipping the mode header and any malformed lines
    silently — best-effort enrichment is preferable to all-or-
    nothing. ParseFile is the disk-read wrapper. parseSegment and
    parseLineCol split out so the grammar stays readable.

  - coverage: projectStats picks segments whose start line falls
    inside [StartLine, EndLine] inclusive. Start-line containment
    matches what `go tool cover -func` produces — Go's cover tool
    scopes each segment to the immediately enclosing block, so
    picking by start-line correctly attributes nested-block
    segments to the right function. Stats expose num_stmt and hit
    plus a Percent() method that returns -1 for zero-statement
    nodes so callers can distinguish "uncovered" from "no
    measurement".

  - coverage: EnrichGraph groups segments by repo-relative file so
    each file is projected once even when the profile lists
    thousands of segments per package. Stamps meta.coverage_pct
    (rounded to 2 decimals) plus meta.coverage = {num_stmt, hit}
    on every KindFunction / KindMethod / KindClosure node it can
    map. Other kinds are deliberately excluded — types, fields,
    and variables have no coverage signal of their own.

  - coverage: stripModulePrefix adapts module-qualified profile
    paths (github.com/foo/bar/pkg/file.go) to repo-relative graph
    paths (pkg/file.go). Always strips a leading ./ so profiles
    built outside a module-aware context (raw `go test
    -coverprofile`) work alongside the standard form. Falling
    through to the unprefixed shape on prefix mismatch keeps the
    enrichment functional when the profile and graph were
    generated against different repo layouts. ReadModulePath
    extracts the module declaration from go.mod so callers don't
    need to hand-wire the prefix.

  - mcp: handleAnalyze gains a coverage case. handleAnalyzeCoverage
    reads the `profile` arg (absolute or relative to RootPath),
    parses, runs ReadModulePath against the same root, and calls
    EnrichGraph. Returns {enriched, segments, profile,
    module_path} so callers can sanity-check the prefix-strip
    worked. Re-runnable: each call re-reads the profile and
    overwrites existing meta, the correct behaviour after a fresh
    test run. Tool description and parameter schema updated to
    advertise the new kind plus the profile arg.

  - tests: 11 cases in coverage — basic profile parsing, malformed-
    line skipping, projectStats with covered + uncovered + outside-
    range segments, no-coverage edge case, EnrichGraph with mixed
    covered/uncovered functions, non-executable-kind exclusion,
    unprefixed profile paths, stripModulePrefix variants,
    ReadModulePath happy path + missing-file fallback, and
    roundTwo. 2 cases in mcp — full handler integration that
    builds a synthetic profile and asserts main=100/helper=0, plus
    rejection of missing-profile-arg requests. Race-enabled tests
    pass.
2026-05-03 02:01:19 +02:00