Merge branch 'main' into feat/copilot-installer-targets
# Conflicts: # CHANGELOG.md
This commit is contained in:
Colby McHenry
@@ -58,6 +58,12 @@ scripts/agent-eval/audit.sh <VERSION> <repo-name> <repo-url> "<question>" <MODE>
|
||||
codegraph-tool calls, duration, **total cost**.
|
||||
- Interactive (`parse-session.mjs`): the `VERDICT: codegraph_explore used Nx |
|
||||
Read N | Grep/Bash N` and `TOKENS:` lines.
|
||||
- Both paths also print the three feedback metrics — residual context occupancy,
|
||||
explore sufficiency, allocation efficiency — and a headless A/B ends with a
|
||||
side-by-side `ARM COMPARISON` table. Report that table, and check its
|
||||
contamination row first: `CLI calls that RETURNED output` > 0 means the arm
|
||||
reached codegraph through Bash and its numbers are void. How to read the rest:
|
||||
`docs/benchmarks/agent-eval-feedback-metrics.md`.
|
||||
|
||||
Lead with cost + tool/Read counts — they are the reliable signals; raw token
|
||||
in/out are confounded by subagent delegation and prompt caching. State whether
|
||||
|
||||
@@ -76,3 +76,6 @@ __tests__/zz-scratch*
|
||||
|
||||
# linux-arm64 kernel cross-build cache (rust:1-bookworm builder)
|
||||
target-linux/
|
||||
.kommandr/kommandr.db
|
||||
.kommandr/kommandr.db-wal
|
||||
.kommandr/kommandr.db-shm
|
||||
|
||||
Binary file not shown.
@@ -11,8 +11,38 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
### New Features
|
||||
|
||||
- Anonymous usage telemetry is now stored entirely on CodeGraph's own first-party infrastructure — no third-party analytics vendor receives any of it, and the endpoint that receives it makes no outbound requests at all. Individual events are deleted after 90 days, leaving only anonymous daily totals. Nothing about what is collected changed, your IP address is still never read or stored, and every off-switch works exactly as before (`codegraph telemetry off`, `CODEGRAPH_TELEMETRY=0`, `DO_NOT_TRACK=1`). `TELEMETRY.md` remains the complete field-by-field list.
|
||||
|
||||
- `codegraph_explore` no longer re-sends source it already returned earlier in the same conversation. A file it has already shown you comes back as a short pointer — the path, the symbols and the exact line range, with confirmation that the file hasn't changed since — and the space that frees is spent on code you haven't seen yet, so a follow-up call covers new ground instead of repeating the last one. If a file was edited in between, its source is always shown again in full. Set `CODEGRAPH_EXPLORE_DEDUP=0` to turn this off.
|
||||
|
||||
- When an agent connects over MCP, CodeGraph now states up front that it indexes 30+ languages — TypeScript/JavaScript, Python, Go, Rust, Java, C#, C/C++, PHP, Ruby, Swift, Kotlin, and more — so agents no longer assume a language isn't supported and skip the graph. (#671)
|
||||
|
||||
- GitHub Copilot is now a supported agent: `codegraph install` can configure Copilot Chat in VS Code (`copilot-vscode`), the GitHub Copilot CLI (`copilot-cli`), and the Copilot plugin in JetBrains IDEs (`copilot-jetbrains`). Installed Copilot surfaces are auto-detected like every other agent, existing MCP server entries in their config files are preserved, and `codegraph uninstall` reverses the setup cleanly. Restart VS Code or your JetBrains IDE after installing so Copilot picks up the server.
|
||||
|
||||
### Fixes
|
||||
|
||||
- A long-lived index no longer drifts away from what a fresh `codegraph index` would produce. When a file gained or lost a symbol, references to that name in files the sync never touched kept pointing at the definition that was correct before the change, and — because nothing distinguished two same-named definitions — the winner could come down to the order files happened to be written, which differs between a full index and a sync. On this project's own repository, replaying 80 commits through `sync` left 5.7% of connections wrong; it is now 1.3%, and the wrong-answers-still-being-asserted half drops by 99.7%. Since call edges are what flow questions follow and what `codegraph_explore` ranks files by, this quietly degraded answers as an index aged, with nothing to indicate it. Syncing is unchanged in speed, and an edit that only changes a function's body does no extra work at all. Set `CODEGRAPH_NO_REBIND=1` to opt out.
|
||||
- `codegraph_explore` now concentrates its answer on the code that actually answers your question instead of spreading it across files that merely share a word with it, so more of the answer arrives in a single call. Thanks @LeDuyViet for the detailed measurements and reproduction. (#1500)
|
||||
- Files only weakly related to your question now come back as a name, symbol and line number instead of spending the answer on their source — name one of them in a follow-up `codegraph_explore` to get it back in full. (#1500)
|
||||
- A generated CRUD or protobuf layer no longer crowds out the hand-written code sitting beside it: generated files are now recognized by the `// Code generated by … DO NOT EDIT.` style banner written at the top of the file, not just by a filename that looks generated. Re-index after upgrading to pick up the new detection. (#1500)
|
||||
- Test and spec files in a repository's top-level `test/` or `spec/` directory are now recognized as such, so they no longer take room from the code you asked about. (#1500)
|
||||
- Generated type-declaration files that announce themselves with a "Generated by … by running …" banner — Cloudflare Wrangler's `worker-configuration.d.ts` is the common one — are now recognized as generated. Previously a file like that could take most of a `codegraph_explore` answer on nothing more than a few common words, pushing the hand-written code you asked about out of the response entirely. Re-index after upgrading to pick up the new detection.
|
||||
- A hand-written type-declaration file — an ambient `.d.ts` of global shims, vendored typings, module augmentation — no longer takes over a `codegraph_explore` answer about how something works. Files like these declare common names (`Body`, `Message`, `ImageMetadata`) and nothing else, so a plainly-worded question could match one strongly enough that it ranked first and crowded the actual handler out of the answer. They are now ranked lower for questions about behaviour, and are still listed by name so one follow-up call fetches them. Asking about a type by name still returns its declaration first, and a shared types module the rest of your code imports is unaffected.
|
||||
- A CodeGraph process that gets force-killed — by the stuck-process watchdog, a crash, or the OS — no longer leaves the database's write-ahead log behind to grow without bound. Previously each killed session stacked more data onto the same log file and nothing ever shrank it, which on machines where sessions were killed regularly could quietly eat tens of gigabytes of disk. The log is now capped, and any oversized leftover is reclaimed automatically the next time the project is opened. Thanks @tiendungdev for the exceptional Windows report that pinned this down. (#1431)
|
||||
- The background server's watchdog no longer kills a healthy server that is just waiting on a slow disk: like indexing already does, it now checks whether the database files are still making progress before concluding the process is stuck. Fewer spurious kills also means fewer leftover write-ahead logs. (#1431)
|
||||
- `codegraph status` now shows the write-ahead log's size next to the database size and warns when killed sessions have left it oversized, and every line in the background server's log now carries a timestamp so kills and restarts can be placed in time. (#1431)
|
||||
- On Windows, the Claude Code prompt hook written by `codegraph install` failed with "command not found" when hooks run through Git Bash, which needs the `.cmd` extension to find the launcher. The installer now writes the platform-correct command, and re-running `codegraph install` (or `codegraph upgrade`) repairs an existing install in place. (#1466)
|
||||
- Python classes used as values — `return SomeSerializer` from a factory method, `handler = SomeClass` aliases, registry dicts and lists, and classes passed as arguments — now produce reference edges in the graph. Previously these idioms were invisible, so on Django and Django REST Framework projects, asking for a serializer's callers or the impact of editing it missed the views that actually use it. Re-index after upgrading to pick up the new edges. (#1478)
|
||||
- When a file changed on disk after its last index sync, `codegraph_node` and `codegraph_explore` could return a different symbol's code under the requested name — current file bytes cut at outdated line positions — while presenting it as verbatim, trustworthy source. This hit hardest on projects queried through `projectPath` (for example, sub-projects of a monorepo), which have no live file watcher to flag pending edits. Both tools now verify each file against the index before showing sliced code: an out-of-date file is either shown whole with its full current source, or its code is withheld with a clear "changed on disk" notice — never served as a wrong slice. A fresh re-index restores normal output automatically. Thanks @inth3shadows for the thorough report and verification passes. (#1474)
|
||||
- A file built around one very long function no longer takes the whole `codegraph_explore` answer for itself — or disappears from it. Previously such a file was shown in full however big it was, which used up the room every file after it needed, and when the function was larger than the entire response the file was dropped without a word. These files now come back as a bounded window on whole lines — the signature and the top of the body, plus the call site when the call path runs through it — with the rest one follow-up `codegraph_explore` away.
|
||||
- `codegraph_explore` no longer lets the first file in an answer spend the room set aside for the files below it, so the rest of the answer still arrives. Previously a large file near the top could quietly use up everything left, and the files ranked under it — each already judged relevant enough to include — were dropped with no source at all; on one question only one of six made it into the answer. Every file now keeps what it was given, and a question that really is about one file still concentrates on that file.
|
||||
- When a `codegraph_explore` answer runs right up against its size limit, it now drops the trailing notes rather than a whole file's source. Previously the last file was cut even though trimming the notes alone would have fit, so a file that had already been read, ranked and rendered was thrown away at the last moment. Across a range of real projects this returns one more file and up to 20% more source per call.
|
||||
- Every file `codegraph_explore` decides to include now actually arrives. A file shown in full could still spend room set aside for files below it — the fix above covered files shown as excerpts but not files shown whole — and the answer's own size bookkeeping under-counted each file's heading, so the answer ran past its limit and a fully prepared file was discarded at the end. A file that no longer fits whole is now shown as excerpts instead of vanishing, and one that overshoots by a little is trimmed to fit rather than dropped.
|
||||
- The list of files an answer could not cover — the "explore these names for their source" pointers — is no longer thrown away when the answer is full. It is now budgeted for and trimmed to fit, so a full answer still tells you what it left out and which names to ask for next, instead of ending with no pointers at all.
|
||||
- When a `codegraph_explore` answer shows a file as excerpts, a large excerpt that no longer fit was dropped entirely instead of being shortened. If the file's first excerpt happened to be a trivial one — an import block, a one-line helper next to the code you asked about — the excerpt carrying the actual answer was the one thrown away, and the file came back with a quarter of the room it had been given. On real projects that meant the top-ranked file delivered a fraction of its share while a far less relevant file took the rest. Excerpts are now shortened to fit, whole method by whole method, and only dropped when what is left is too small to hold anything readable.
|
||||
- When you name a symbol in a `codegraph_explore` query, its definition now actually comes back. Two cases previously lost it. If the symbols you named don't call one another — sibling functions inside the same factory or module are the everyday example — CodeGraph stopped treating them as symbols you had asked for, and answered with whatever sat at the top of their file instead; on one 1,400-line file that meant a same-stem `QueuedMessage` interface on line 70 came back while the `queueMessage` function on line 1087 did not. And when an answer had to be trimmed to fit, it was trimmed from the bottom of the file down, so a symbol near the end of a long file was always the first thing cut. Trimming now protects the definitions you named wherever they sit in the file.
|
||||
- The blast-radius section of `codegraph_explore` flagged "no covering tests found" whenever no test called a symbol directly — falsely branding helpers that tests exercise through their callers as untested (about 40% of flagged symbols in a measured sample). The check now follows caller chains up to 3 hops and reports indirect coverage as "tested via callers"; when nothing is found it states exactly what was checked instead of an unconditional warning. Thanks @inth3shadows for measuring the false-positive rate. (#1475)
|
||||
|
||||
## [1.5.0] - 2026-07-21
|
||||
|
||||
# ⚡ The Rust engine release — with near-instant sync
|
||||
|
||||
@@ -138,6 +138,8 @@ For each **language × framework**, validate on **small, medium, and large** rea
|
||||
1. **Pick the canonical flow** for the framework ("how does X reach Y": state→render, request→handler→view, query→SQL, action→reducer→store…).
|
||||
2. **Deterministic probes** (`scripts/agent-eval/probe-{node,explore}.mjs` against the built `dist/`): `codegraph_explore` with the flow's symbol names connects from→to end-to-end with no break (its Flow section shows the path); **no node explosion** (`select count(*) from nodes` stable before/after re-index); synthesized-edge **precision** spot-check (`select … where provenance='heuristic'`).
|
||||
3. **Agent A/B** (`scripts/agent-eval/run-all.sh <repo> "<Q>"`): with vs without codegraph, **≥2 runs/arm** (run-to-run variance is large — never conclude from n=1). Record **duration, total tool calls, Read, Grep**. Optional forced-Read-0 sufficiency proof via the block-read hook (`scripts/agent-eval/hook-settings.json`).
|
||||
- **Every run also reports three feedback metrics** — residual context occupancy, explore sufficiency (what the agent did NEXT after each explore), and allocation efficiency (share of returned bytes the answer cited) — under each run, plus a side-by-side arm table (`compare-arms.mjs`). Entry point: `docs/benchmarks/agent-eval-feedback-metrics.md`. Reading them: `Read a file we returned` is an allocation miss, `Read a file we did NOT return`/`Grep` is recall; allocation efficiency is **relative** (attribution is by citation) so it is only valid between builds on the same question; occupancy *shares* are Claude Code / 200k and don't transfer to another host — the arm ratio does.
|
||||
- **The `codegraph` CLI is blocked in every arm** (`no-cli-shim.sh`: sanitized PATH + a PreToolUse hook, shared by both harnesses). Without it 14 of 15 without-arm runs in one 7-repo pass reached codegraph through Bash. Check the contamination row before believing any number: `CLI calls that RETURNED output` > 0 invalidates the run (in a new-vs-baseline A/B it silently drops calls from all three metrics, since a CLI explore is not a tool call).
|
||||
- **Model policy — every A/B arm runs Claude with `--model sonnet --effort high`. Always. Never Opus/Fable.** All `scripts/agent-eval/*.sh` default to this (`MODEL`/`EFFORT` env override exists — don't raise it without an explicit reason from the maintainer). Two reasons, and the second matters more than cost: (a) Sonnet doesn't burn tokens; (b) **Sonnet is the deliberate floor model** — codegraph's real users attach it to whatever agent they already run (Cursor Composer, Gemini, etc.), so we validate on a "dumber" model on purpose: a stronger model's tool-use covers up the salience/sufficiency problems a weaker one exposes. An affordance that lands on Sonnet generalizes up to every host; one that only works on Opus/Fable doesn't generalize down to the agents most users actually have. Both arms always use the same model.
|
||||
- **MCP attach is a startup-latency issue, not a hard block.** On a multi-step task the agent dives into Read/grep before codegraph finishes its ~2-3s startup (worse when the eval is itself run nested inside a Claude session, under CPU contention), so it runs with no codegraph. Fix: **pre-warm a persistent daemon** for the target (`CODEGRAPH_DAEMON_IDLE_TIMEOUT_MS` high; spawn `serve --mcp --path <target> </dev/null &`; wait for `.codegraph/daemon.sock`) **and skip the startup re-exec** (`CODEGRAPH_WASM_RELAUNCHED=1`) so claude connects before the agent's first turn. Don't trust claude's `init` snapshot — it can read `status:"pending"` / 0 tools even when it then connects; judge by actual codegraph usage in `parse-run.mjs`'s `by type`. To isolate a change — **new-build vs baseline-build, both codegraph-on** (vs run-all.sh's with-vs-without) — use `scripts/agent-eval/ab-new-vs-baseline.sh <indexed-repo> "<task>" [baseline-ref]` (it bakes in the pre-warm).
|
||||
4. **Pass bar:** a normal flow question reaches **~0 Read/Grep within the repo's explore-call budget**, runs **faster** than without-codegraph, and shows **no regression on a control repo**. Record the numbers in `docs/design/dynamic-dispatch-coverage-playbook.md` (the coverage matrix).
|
||||
|
||||
@@ -193,47 +193,51 @@ When an AI agent needs to understand code — to answer a question or make a cha
|
||||
|
||||
<img width="1536" height="1024" alt="token-cost-savings-scale" src="https://github.com/user-attachments/assets/eb74a11a-a3ab-4b01-80a6-19f78352ae8e" />
|
||||
|
||||
> **A note on cost:** CodeGraph's win on *every* codebase is precision — the agent stops crawling files and answers from the graph. On current models that precision is also a large direct saving: the 2026-07 re-validation measured **60% lower cost and 69% fewer tokens on average** across the seven benchmark repos, because a strong model *without* the graph burns millions of tokens re-deriving structure. The savings scale with repo size and tangle — dramatic on VS-Code-class trees, modest on a 100-file project — and compound across a team's daily agent usage.
|
||||
> **A note on cost:** CodeGraph's win on *every* codebase is precision — the agent stops crawling files and answers from the graph. On current models that precision is also a large direct saving: the 2026-08 re-measurement, on a harness that blocks the CLI in both arms, put it at **44% lower cost and 62% fewer tokens on average** across the seven benchmark repos, because a strong model *without* the graph burns its budget re-deriving structure. Cost tracks how much *discovery* a question demands more than raw repo size: 57–78% on questions the file-reading agent needed 28–43 tool calls to answer, near-even where it got there in 7.
|
||||
|
||||
> **A note on context:** the numbers above measure *throughput* — tokens processed, tools called, dollars spent to reach one answer. They don't measure what is still sitting in your context window afterward, and on that axis CodeGraph costs **more**, not less. Across the same seven repos in multi-turn sessions, CodeGraph's responses leave about **80% more retrieval context resident** at the end of a session than a file-reading agent's do — on VS Code, 67k tokens against 18k. The mechanism is the same one that makes it fast: CodeGraph returns one dense, verbatim payload that answers the question and then stays in the window, where a grep-and-read agent churns through many small results that get evicted. Fewer tokens *processed* and a larger persistent *footprint* are both real at once. If you run long sessions in a small window, budget for it. Measured per-repo: [`docs/benchmarks/residual-context-occupancy.md`](docs/benchmarks/residual-context-occupancy.md).
|
||||
|
||||
### Benchmark Results
|
||||
|
||||
Tested across **7 real-world open-source codebases** spanning 7 languages, comparing an agent (Claude Code, headless) answering one architecture question **with** and **without** CodeGraph, at the **median of 4 runs per arm**. _Re-validated 2026-07-21 on **Claude Opus 4.8** against the current build — the Rust kernel plus this cycle's resolution overhaul._
|
||||
Tested across **7 real-world open-source codebases** spanning 7 languages, comparing an agent (Claude Code, headless) answering one architecture question **with** and **without** CodeGraph, at the **median of 4 runs per arm**. _Re-measured 2026-08-05 on **Claude Opus 4.8** against the current build, on a harness that blocks the `codegraph` CLI in **both** arms — contamination row: 0 of 28 without-arm runs._
|
||||
|
||||
> **The universal win — every repo, every size: 89% fewer tool calls · 60% cheaper · 69% fewer tokens · file reads cut to zero on all seven repos.**
|
||||
> **The universal win — every repo, every size: 88% fewer tool calls · 53% faster · 62% fewer tokens · 44% cheaper · file reads cut to zero on all seven repos.**
|
||||
|
||||
With the index available, the agent answers from a couple of `codegraph_explore` calls and stops. Without it, the agent burns its budget on discovery — up to **57 tool calls and 4.3M tokens** re-deriving what the graph already knew. The **Time** column averages 20% faster but is the noisiest metric: on two small repos a strong model's raw grep loop finishes the wall-clock race sooner while still spending 5–10× the tokens and money — noted per-row below.
|
||||
With the index available, the agent answers from one to four `codegraph_explore` calls and stops. Without it, the agent burns its budget on discovery — up to **43 tool calls and 19 file reads** re-deriving what the graph already knew. Every repo was faster with CodeGraph in this measurement — by 35% on the narrowest question, by 3.6× on the widest.
|
||||
|
||||
| Codebase | Language | Tool calls | Time | File reads | Tokens | Cost |
|
||||
|----------|----------|------------|------|------------|--------|------|
|
||||
| **VS Code** | TypeScript · ~11k files | **2 vs 40** | **5× faster** (41s vs 3m 24s) | **0** vs 17 | 83% fewer | 75% cheaper |
|
||||
| **Excalidraw** | TypeScript · ~640 | 3 vs 55 | 36s vs 23s¹ | **0** vs 24 | 89% fewer | 78% cheaper |
|
||||
| **Django** | Python · ~3k | **2 vs 29** | 38% faster | **0** vs 16 | 78% fewer | 69% cheaper |
|
||||
| **Tokio** | Rust · ~790 | 3 vs 57 | 65% faster | **0** vs 15 | 91% fewer | 86% cheaper |
|
||||
| **OkHttp** | Java · ~645 | 1 vs 5 | 10% faster | **0** vs 1 | 33% fewer | ~even² |
|
||||
| **Gin** | Go · ~110 | 3 vs 10 | 57% faster | **0** vs 4 | 18% fewer | 41% cheaper |
|
||||
| **Alamofire** | Swift · ~110 | 3 vs 53 | 49s vs 31s¹ | **0** vs 18 | 90% fewer | 86% cheaper |
|
||||
| **VS Code** | TypeScript · ~11k files | **2 vs 28** | **2.2× faster** (58s vs 2m 10s) | **0** vs 12 | 77% fewer | 71% cheaper |
|
||||
| **Excalidraw** | TypeScript · ~640 | **2 vs 43** | **3.6× faster** (45s vs 2m 42s) | **0** vs 18 | 84% fewer | 78% cheaper |
|
||||
| **Django** | Python · ~3k | 3 vs 14 | 35% faster (54s vs 1m 23s) | **0** vs 8.5 | 41% fewer | 13% cheaper¹ |
|
||||
| **Tokio** | Rust · ~790 | 3 vs 29 | **2.6× faster** (1m 3s vs 2m 43s) | **0** vs 19 | 65% fewer | 64% cheaper |
|
||||
| **OkHttp** | Java · ~645 | 1 vs 6 | 43% faster (33s vs 58s) | **0** vs 2 | 54% fewer | 21% cheaper |
|
||||
| **Gin** | Go · ~110 | 1 vs 7 | 39% faster (28s vs 46s) | **0** vs 4 | 52% fewer | ~even¹ |
|
||||
| **Alamofire** | Swift · ~110 | 4 vs 33 | **2.6× faster** (54s vs 2m 22s) | **0** vs 16.5 | 59% fewer | 57% cheaper |
|
||||
|
||||
<sub>¹ The small-repo floor effect: Opus 4.8 greps small trees fast enough to win wall-clock while spending ~5–10× the tokens and ~4–7× the cost — the with-arm still answers from zero file reads. ² OkHttp's without-arm got lucky in 5 calls; the with-arm answered in 1 call for ~$0.03 more. **File reads** = median files opened — the surgical-context win in one column: the agent never reads a file on any of the seven repos when CodeGraph is present.</sub>
|
||||
<sub>¹ Cost tracks how much *discovery* the question demanded, which is why it varies far more than the other columns: 57–78% on repos where the file-reading arm needed 28–43 tool calls, but only 13% on Django and even on Gin, where it got there in 14 and 7. The with-arm still answered in 3 and 1 calls with zero file reads. **File reads** = median files opened — the surgical-context win in one column: the agent never reads a file on any of the seven repos when CodeGraph is present.</sub>
|
||||
|
||||
<details>
|
||||
<summary><strong>Per-repo breakdown — WITH vs WITHOUT (median of 4)</strong></summary>
|
||||
|
||||
| Codebase | Metric | WITH cg | WITHOUT cg |
|
||||
|---|---|---|---|
|
||||
| **VS Code** | Time / Tools / Tokens / Cost | 41s / 2 / 265k / $0.36 | 3m 24s / 40 / 1.5M / $1.41 |
|
||||
| **Excalidraw** | Time / Tools / Tokens / Cost | 36s / 3 / 324k / $0.40 | 23s / 55 / 2.9M / $1.81 |
|
||||
| **Django** | Time / Tools / Tokens / Cost | 42s / 2 / 254k / $0.35 | 1m 8s / 29 / 1.2M / $1.13 |
|
||||
| **Tokio** | Time / Tools / Tokens / Cost | 46s / 3 / 386k / $0.44 | 2m 11s / 57 / 4.3M / $3.04 |
|
||||
| **OkHttp** | Time / Tools / Tokens / Cost | 27s / 1 / 156k / $0.23 | 30s / 5 / 233k / $0.20 |
|
||||
| **Gin** | Time / Tools / Tokens / Cost | 30s / 3 / 246k / $0.27 | 1m 10s / 10 / 300k / $0.46 |
|
||||
| **Alamofire** | Time / Tools / Tokens / Cost | 49s / 3 / 316k / $0.35 | 31s / 53 / 3.1M / $2.51 |
|
||||
| **VS Code** | Time / Tools / Tokens / Cost | 58s / 2 / 155k / $0.53 | 2m 10s / 28 / 670k / $1.80 |
|
||||
| **Excalidraw** | Time / Tools / Tokens / Cost | 45s / 2 / 156k / $0.54 | 2m 42s / 43 / 991k / $2.43 |
|
||||
| **Django** | Time / Tools / Tokens / Cost | 54s / 3 / 183k / $0.55 | 1m 23s / 14 / 309k / $0.63 |
|
||||
| **Tokio** | Time / Tools / Tokens / Cost | 1m 3s / 3 / 201k / $0.66 | 2m 43s / 29 / 573k / $1.83 |
|
||||
| **OkHttp** | Time / Tools / Tokens / Cost | 33s / 1 / 107k / $0.39 | 58s / 6 / 230k / $0.50 |
|
||||
| **Gin** | Time / Tools / Tokens / Cost | 28s / 1 / 87k / $0.31 | 46s / 7 / 180k / $0.31 |
|
||||
| **Alamofire** | Time / Tools / Tokens / Cost | 54s / 4 / 209k / $0.54 | 2m 22s / 33 / 505k / $1.27 |
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Full benchmark details</strong></summary>
|
||||
|
||||
**Methodology.** Each arm is `claude -p` (Claude Opus 4.8) run headlessly against the repo with `--strict-mcp-config`: **WITH** = CodeGraph's MCP server enabled, **WITHOUT** = an empty MCP config. Built-in Read/Grep/Bash stay available to both. Same question per repo, **4 runs per arm, median reported**. Cost = the run's `total_cost_usd`; Tokens = total tokens processed (input incl. cached + output); Time = wall-clock; Tool calls = every tool invocation, including those inside any sub-agents the model spawns. Repos cloned at `--depth 1` and indexed by the same CodeGraph build that served them. Re-validated 2026-07-21 on the current build (native Rust kernel, adaptive parallel resolution, scoped sync).
|
||||
**Methodology.** Each arm is `claude -p` (Claude Opus 4.8, `claude-opus-4-8`) run headlessly against the repo with `--strict-mcp-config`: **WITH** = CodeGraph's MCP server enabled, **WITHOUT** = an empty MCP config. Built-in Read/Grep/Bash stay available to both. Same question per repo, **4 runs per arm, median reported**. Cost = the run's `total_cost_usd`; Tokens = total tokens processed, summed per assistant turn (input incl. cache reads + cache creation + output); Time = wall-clock; Tool calls = every tool invocation, including those inside any sub-agents the model spawns. Repos cloned at `--depth 1` and indexed by the same CodeGraph build that served them. Re-measured 2026-08-05 on the current build.
|
||||
|
||||
**The `codegraph` CLI is blocked in both arms.** A sanitized `PATH` plus a `PreToolUse` hook denies any Bash invocation of the CLI, in the WITHOUT arm as well as the WITH arm. This matters: without that block the control arm is not a control. On an unblocked harness we measured the WITHOUT agent finding the CLI on `PATH` and reaching CodeGraph through Bash in **26 of 28 runs** — which distorts the comparison in both directions, since a CLI call is not counted as a tool call and its output still enters the window. Earlier published figures were produced without this block. In the run reported above, all 28 WITHOUT runs attempted the CLI and **all 28 were blocked — 0 contaminated**.
|
||||
|
||||
**Queries:**
|
||||
| Codebase | Query |
|
||||
|
||||
+25
-8
@@ -70,8 +70,8 @@ per-call event stream, and nothing is sent in real time.
|
||||
- **No source code.** No file paths, file names, directory names, repository names or
|
||||
URLs, symbol names, search queries, or anything else derived from the contents of an
|
||||
indexed project.
|
||||
- **No IP addresses.** The ingest endpoint never reads, logs, or forwards the client IP,
|
||||
and IP discarding is enabled at the analytics backend on top of that. No geolocation.
|
||||
- **No IP addresses.** The ingest endpoint never reads, logs, or stores the client IP —
|
||||
and there is no analytics vendor downstream that could. No geolocation.
|
||||
- **No fingerprinting.** The machine ID is a random UUID stored in
|
||||
`~/.codegraph/telemetry.json` — delete that file (or run `codegraph telemetry off`,
|
||||
then `on`) and the old ID is gone forever, with no way to reconnect it.
|
||||
@@ -81,12 +81,29 @@ per-call event stream, and nothing is sent in real time.
|
||||
|
||||
Events POST to `telemetry.getcodegraph.com` — a first-party endpoint whose complete
|
||||
source lives in [`telemetry-worker/`](telemetry-worker/) in this repository. It validates
|
||||
every event and property against the allowlist above (anything else is dropped), strips
|
||||
IPs, rate-limits, and forwards to a managed analytics store (PostHog, US region) as
|
||||
anonymous events. Sends are fire-and-forget with a short timeout: offline or air-gapped
|
||||
machines buffer a bounded local file (256 KB cap) and never retry-loop, log errors, or
|
||||
slow a command down. Telemetry never adds latency to MCP tool calls — recording is an
|
||||
in-memory counter.
|
||||
every event and property against the allowlist above (anything else is dropped), never
|
||||
reads the client IP, and rate-limits per machine ID. Sends are fire-and-forget with a
|
||||
short timeout: offline or air-gapped machines buffer a bounded local file (256 KB cap)
|
||||
and never retry-loop, log errors, or slow a command down. Telemetry never adds latency to
|
||||
MCP tool calls — recording is an in-memory counter.
|
||||
|
||||
## Where it is stored
|
||||
|
||||
Accepted events are written to **our own database on Cloudflare** (D1) and go nowhere
|
||||
else. **No third-party analytics vendor receives any of this data**, because the ingest
|
||||
endpoint makes no outbound requests at all — its source is the entire path your events
|
||||
take, and there is nothing after it. This is a stronger guarantee than a promise not to
|
||||
share: there is no second party to share with.
|
||||
|
||||
What is kept is checkable rather than asserted. The storage schema —
|
||||
[`telemetry-worker/migrations/0001_init.sql`](telemetry-worker/migrations/0001_init.sql),
|
||||
checked in beside the endpoint that writes it — is the complete list of what a row can
|
||||
hold, with a comment on every column.
|
||||
|
||||
Individual events are **deleted after 90 days**. What outlives them is anonymous daily
|
||||
totals: counts per day of things like operating system, version, and language, plus which
|
||||
days each machine ID was active so returning-user numbers survive. No event details, and
|
||||
still nothing that identifies a person or a codebase.
|
||||
|
||||
The engineering contract behind all of this — including the rule that schema changes must
|
||||
update this page, the client, and the public endpoint in one PR — is in
|
||||
|
||||
@@ -16,7 +16,7 @@ import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import { DatabaseConnection } from '../src/db';
|
||||
import { QueryBuilder } from '../src/db/queries';
|
||||
import { runMigrations, getCurrentVersion } from '../src/db/migrations';
|
||||
import { runMigrations, getCurrentVersion, CURRENT_SCHEMA_VERSION } from '../src/db/migrations';
|
||||
import { Node, Edge } from '../src/types';
|
||||
|
||||
function makeNode(id: string, name = id): Node {
|
||||
@@ -344,7 +344,11 @@ describe('migration v6: dedup edges + add identity index on upgrade (#1034)', ()
|
||||
runMigrations(raw, 5);
|
||||
|
||||
expect(count()).toBe(2); // duplicate collapsed, the distinct `calls` edge kept
|
||||
expect(getCurrentVersion(raw)).toBe(8);
|
||||
// Migrations ran to completion. Tracked against the constant, not a
|
||||
// literal, so adding a migration doesn't require editing this assertion —
|
||||
// and so replaying every migration over a current-schema database (which
|
||||
// is what this test does) stays covered as new ones land.
|
||||
expect(getCurrentVersion(raw)).toBe(CURRENT_SCHEMA_VERSION);
|
||||
const idx = raw
|
||||
.prepare("SELECT name FROM sqlite_master WHERE type='index' AND name='idx_edges_identity'")
|
||||
.get();
|
||||
|
||||
@@ -0,0 +1,301 @@
|
||||
/**
|
||||
* Regression fixture for GitHub issue #1500 / epic CG-1 — relevance-proportional
|
||||
* explore budget allocation.
|
||||
*
|
||||
* The reporter's repo is a Go service whose GENERATED FKIT CRUD layer sits beside
|
||||
* the hand-written use-case that does the real work. Asking an architecture
|
||||
* question that doesn't name the exact use-case ("how does payroll cycle create
|
||||
* and calculate payslips?") spends the explore envelope on the generated CRUD,
|
||||
* because the generated layer name-collides on every term in the question while
|
||||
* the hand-written workflow is one big file that gets clipped.
|
||||
*
|
||||
* `__tests__/fixtures/payroll-go/` reproduces that shape permanently. This suite
|
||||
* is in two halves:
|
||||
*
|
||||
* 1. **Fixture shape** — green today. These pin the properties the fixture must
|
||||
* keep for the gate below to mean anything: the generated/hand-written split
|
||||
* (including the ordinary-named generated files only a CONTENT header betrays,
|
||||
* which is the #1500 case), the deliberate name collisions, and the
|
||||
* runPayrollCycleAll → BuildPayslip → Upsert chain resolving end-to-end. If
|
||||
* the fixture rots, these fail first and say so.
|
||||
*
|
||||
* 2. **Budget allocation** — the gate. CG-10 (relevance scoring) closed most of
|
||||
* it: the generated CRUD now ranks and delivers BELOW the hand-written
|
||||
* workflow, and those assertions are live regressions. What remains is
|
||||
* `it.fails`, which DOCUMENTS THE PART STILL OPEN — vitest passes an
|
||||
* `it.fails` test only while its body throws, so it goes RED the moment
|
||||
* CG-12's proportional byte allocation lands.
|
||||
* **When it goes red, delete the `.fails` — do not delete the test.**
|
||||
*
|
||||
* The same assertions run outside vitest, against the built dist and with the
|
||||
* full CG-4 per-file diagnostic, via `node scripts/agent-eval/probe-allocation.mjs`
|
||||
* (declared in `scripts/agent-eval/allocation-fixtures.json`).
|
||||
*/
|
||||
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import CodeGraph from '../src/index';
|
||||
import { ToolHandler, getExploreOutputBudget } from '../src/mcp/tools';
|
||||
import { attributeSourceBytes } from '../src/mcp/explore-diagnostics';
|
||||
import { isGeneratedFile, hasGeneratedHeader } from '../src/extraction/generated-detection';
|
||||
|
||||
const FIXTURE_SRC = path.join(__dirname, 'fixtures', 'payroll-go');
|
||||
|
||||
/** The question a newcomer asks — names none of the symbols that answer it. */
|
||||
const QUERY = 'how does payroll cycle create and calculate payslips?';
|
||||
|
||||
/** The hand-written workflow: what the query is actually about. */
|
||||
const ANSWER_PREFIXES = [
|
||||
'internal/usecase/',
|
||||
'internal/store/',
|
||||
'internal/transport/',
|
||||
'internal/domain/',
|
||||
'cmd/',
|
||||
];
|
||||
/** The generated CRUD/DTO layer: what wins the envelope today. */
|
||||
const GENERATED_PREFIX = 'internal/gen/';
|
||||
|
||||
const startsWithAny = (p: string, prefixes: string[]) => prefixes.some((x) => p.startsWith(x));
|
||||
|
||||
describe('#1500 — generated Go CRUD beside a hand-written payroll workflow', () => {
|
||||
let testDir: string;
|
||||
let cg: CodeGraph;
|
||||
let handler: ToolHandler;
|
||||
let response: string;
|
||||
/** Delivered source bytes per file, attributed from the final response. */
|
||||
let bytes: Map<string, number>;
|
||||
|
||||
beforeAll(async () => {
|
||||
testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-1500-'));
|
||||
fs.cpSync(FIXTURE_SRC, testDir, { recursive: true });
|
||||
// A stray index in the checked-in tree would be copied in and reused.
|
||||
fs.rmSync(path.join(testDir, '.codegraph'), { recursive: true, force: true });
|
||||
|
||||
cg = CodeGraph.initSync(testDir);
|
||||
await cg.indexAll();
|
||||
handler = new ToolHandler(cg);
|
||||
|
||||
const result = await handler.execute('codegraph_explore', { query: QUERY });
|
||||
response = result.content?.[0]?.text ?? '';
|
||||
bytes = attributeSourceBytes(response);
|
||||
}, 120_000);
|
||||
|
||||
afterAll(() => {
|
||||
if (cg) cg.destroy();
|
||||
if (testDir && fs.existsSync(testDir)) fs.rmSync(testDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
// ── 1. Fixture shape ──────────────────────────────────────────────────────
|
||||
|
||||
describe('fixture shape', () => {
|
||||
it('indexes as a Go project with both layers present', () => {
|
||||
const files = cg.getFiles().map((f) => f.path);
|
||||
expect(files.filter((p) => p.endsWith('.go')).length).toBeGreaterThanOrEqual(15);
|
||||
expect(files.some((p) => p.startsWith(GENERATED_PREFIX))).toBe(true);
|
||||
expect(files.some((p) => p.startsWith('internal/usecase/'))).toBe(true);
|
||||
});
|
||||
|
||||
it('flags every generated file and no hand-written one', () => {
|
||||
for (const file of cg.getFiles()) {
|
||||
expect(file.generated, `${file.path} generated flag`).toBe(
|
||||
file.path.startsWith(GENERATED_PREFIX),
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
it('carries generated files that ONLY a content header betrays — the #1500 case', () => {
|
||||
// Half the generated tree has ordinary names (`payslip.go`, `store.go`).
|
||||
// Path-only detection misses them; the CG-5 content check is what catches
|
||||
// them. Without these the fixture would be a .pb.go fixture, not a #1500 one.
|
||||
const contentOnly = [
|
||||
'internal/gen/fkit/payroll/payslip.go',
|
||||
'internal/gen/fkit/payroll/payroll_cycle.go',
|
||||
'internal/gen/fkit/payroll/store.go',
|
||||
'internal/gen/fkit/payroll/calculate.go',
|
||||
'internal/gen/fkit/payroll/dto.go',
|
||||
'internal/gen/fkit/employee/employee.go',
|
||||
'internal/gen/fkit/timesheet/timesheet.go',
|
||||
];
|
||||
for (const rel of contentOnly) {
|
||||
const source = fs.readFileSync(path.join(testDir, rel), 'utf-8');
|
||||
expect(isGeneratedFile(rel), `${rel} must NOT be detectable by path`).toBe(false);
|
||||
expect(hasGeneratedHeader(source), `${rel} must be detectable by header`).toBe(true);
|
||||
expect(cg.getFile(rel)?.generated, `${rel} indexed flag`).toBe(true);
|
||||
}
|
||||
// …beside the conventional path-detectable ones, so both channels are covered.
|
||||
expect(isGeneratedFile('internal/gen/payrollpb/payroll.pb.go')).toBe(true);
|
||||
});
|
||||
|
||||
it('collides the generated layer with the hand-written one by name', () => {
|
||||
// A naive scorer sees two BuildPayslips and two Upserts and has no reason
|
||||
// to prefer the one that implements the business rule.
|
||||
for (const name of ['BuildPayslip', 'Upsert', 'Store']) {
|
||||
const files = new Set(cg.getNodesByName(name).map((n) => n.filePath));
|
||||
expect([...files].some((p) => p.startsWith(GENERATED_PREFIX)), `${name} generated`).toBe(true);
|
||||
expect([...files].some((p) => !p.startsWith(GENERATED_PREFIX)), `${name} hand-written`).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it('resolves the hand-written workflow chain end-to-end in the graph', () => {
|
||||
const calleesOf = (name: string, file: string) => {
|
||||
const node = cg.getNodesByName(name).find((n) => n.filePath === file);
|
||||
expect(node, `${name} in ${file}`).toBeTruthy();
|
||||
return cg
|
||||
.getOutgoingEdges(node!.id)
|
||||
.filter((e) => e.kind === 'calls')
|
||||
.map((e) => cg.getNode(e.target))
|
||||
.filter((n): n is NonNullable<typeof n> => !!n);
|
||||
};
|
||||
|
||||
// handler → use-case
|
||||
expect(
|
||||
calleesOf('RunCycle', 'internal/transport/httpapi/payroll_handler.go')
|
||||
.some((n) => n.name === 'RunCycle' && n.filePath === 'internal/usecase/payroll/cycle.go'),
|
||||
).toBe(true);
|
||||
|
||||
// use-case → the workflow
|
||||
expect(
|
||||
calleesOf('RunCycle', 'internal/usecase/payroll/cycle.go')
|
||||
.some((n) => n.name === 'runPayrollCycleAll'),
|
||||
).toBe(true);
|
||||
|
||||
// the workflow → build + persist
|
||||
const workflow = calleesOf('runPayrollCycleAll', 'internal/usecase/payroll/cycle.go');
|
||||
expect(
|
||||
workflow.some((n) => n.name === 'BuildPayslip' && n.filePath === 'internal/usecase/payroll/payslip_builder.go'),
|
||||
'runPayrollCycleAll must reach the hand-written BuildPayslip',
|
||||
).toBe(true);
|
||||
expect(workflow.some((n) => n.name === 'Upsert'), 'runPayrollCycleAll must reach an Upsert').toBe(true);
|
||||
});
|
||||
|
||||
it('routes an HTTP entry point into the workflow', () => {
|
||||
const router = cg.getNodesInFile('internal/transport/httpapi/router.go');
|
||||
expect(router.some((n) => n.kind === 'route' || n.name === 'NewRouter')).toBe(true);
|
||||
});
|
||||
|
||||
it('sizes the two layers so the size-driven render split actually bites', () => {
|
||||
// The mechanism the epic is about: a small file ships WHOLE, a large one
|
||||
// falls through to clipped clusters. The workflow file must stay above the
|
||||
// whole-file window and the generated files below it, or the fixture stops
|
||||
// reproducing anything.
|
||||
const lines = (rel: string) => fs.readFileSync(path.join(testDir, rel), 'utf-8').split('\n').length;
|
||||
expect(lines('internal/usecase/payroll/cycle.go')).toBeGreaterThan(220);
|
||||
for (const rel of ['internal/gen/fkit/payroll/payslip.go', 'internal/gen/fkit/payroll/payroll_cycle.go']) {
|
||||
expect(lines(rel)).toBeLessThan(220);
|
||||
}
|
||||
});
|
||||
|
||||
it('answers the query at all', () => {
|
||||
expect(response.length).toBeGreaterThan(1000);
|
||||
expect(bytes.size).toBeGreaterThan(0);
|
||||
});
|
||||
});
|
||||
|
||||
// ── 2. Budget allocation — the open bug ───────────────────────────────────
|
||||
|
||||
describe('budget allocation', () => {
|
||||
const share = (predicate: (p: string) => boolean) => {
|
||||
let total = 0;
|
||||
for (const [file, n] of bytes) if (predicate(file)) total += n;
|
||||
return total / response.length;
|
||||
};
|
||||
const answerShare = () => share((p) => startsWithAny(p, ANSWER_PREFIXES));
|
||||
const generatedShare = () => share((p) => p.startsWith(GENERATED_PREFIX));
|
||||
|
||||
/**
|
||||
* BASELINE 2026-08-03, BEFORE CG-10 (very-tiny tier, 13,000-char budget):
|
||||
* 23,020 chars allocated, cut to 16,011 by the 19,500 hard ceiling. The
|
||||
* generated CRUD delivered 57.4%; the hand-written layer 25.6%, all of it
|
||||
* domain types. `cycle.go` was allocated the single largest slice (7,052
|
||||
* chars, 30.6%) and delivered ZERO — the ceiling dropped its whole section —
|
||||
* so runPayrollCycleAll, the hand-written BuildPayslip and the real Upsert
|
||||
* never reached the agent.
|
||||
*
|
||||
* AFTER CG-10 (relevance scoring): the generated files rank #3/#4 instead of
|
||||
* #1/#2 — kind-weighted scoring plus a generated rank PENALTY on both the
|
||||
* score and the graph mass, rather than the old tiebreak-at-equal-score.
|
||||
* `cycle.go` now delivers 38.9% and the generated layer 23.5%. Four of the
|
||||
* five gates below are green and are now live regressions.
|
||||
*
|
||||
* AFTER CG-12 (score-proportional allocation): every file's share of the
|
||||
* envelope is reserved before anything renders, and a file under 15% of the
|
||||
* top weight gets no source at all — so the two generated files cliff to
|
||||
* pointers, hand their `maxFiles` slots to the hand-written store and
|
||||
* builder, and the answer group takes ~79% with the generated layer at 0%.
|
||||
* `func (s *Service) BuildPayslip` — the "calculate" half of the question —
|
||||
* finally reaches the agent. All gates below are live regressions now.
|
||||
*/
|
||||
it('CG-10 GATE: concentrates the envelope on the hand-written workflow', () => {
|
||||
expect(answerShare()).toBeGreaterThanOrEqual(0.55);
|
||||
});
|
||||
|
||||
it('CG-10 GATE: does not spend the envelope on the generated CRUD', () => {
|
||||
expect(generatedShare()).toBeLessThanOrEqual(0.25);
|
||||
});
|
||||
|
||||
it('CG-10 GATE: ranks the generated CRUD below the hand-written workflow', () => {
|
||||
// The #1500 report in one assertion: before CG-10 the generated layer both
|
||||
// outscored AND out-delivered the use-case that implements the business rule.
|
||||
expect(answerShare()).toBeGreaterThan(generatedShare());
|
||||
});
|
||||
|
||||
it('CG-10 GATE: delivers the workflow file it allocated the most bytes to', () => {
|
||||
expect(bytes.get('internal/usecase/payroll/cycle.go') ?? 0).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('CG-10 GATE: puts the hand-written chain in the response, not its generated twin', () => {
|
||||
// Bare `Upsert` also matches the generated collision — these needles are
|
||||
// unique to the hand-written chain.
|
||||
expect(response).toContain('runPayrollCycleAll');
|
||||
expect(response).toContain('s.store.Upsert(ctx, slip)');
|
||||
});
|
||||
|
||||
it('CG-12 GATE: delivers the calculation the question asks about', () => {
|
||||
// `payslip_builder.go` ranks #6 and the tier's maxFiles is 4 — it reaches
|
||||
// the response only because the two generated files cliff to pointers
|
||||
// WITHOUT consuming a slot. That slot hand-off is the CG-12 mechanism.
|
||||
expect(bytes.get('internal/usecase/payroll/payslip_builder.go') ?? 0).toBeGreaterThan(0);
|
||||
expect(response).toContain('func (s *Service) BuildPayslip');
|
||||
});
|
||||
|
||||
it('CG-12 GATE: withholds the generated CRUD bytes but still names it', () => {
|
||||
// A cliffed file costs ~100 chars instead of ~4,500, and stays one
|
||||
// follow-up explore away — withholding is only cheap if it stays nameable.
|
||||
expect(bytes.get('internal/gen/fkit/payroll/payslip.go') ?? 0).toBe(0);
|
||||
expect(response).toContain('**Not shown above — explore these names for their source**');
|
||||
expect(response).toMatch(/internal\/gen\/fkit\/payroll\/payslip\.go: \w+:\d+/);
|
||||
});
|
||||
|
||||
it('CG-14 GATE: holds the response inside the hard ceiling under real pressure', () => {
|
||||
// This fixture is the stress case for the ceiling, not just for the split:
|
||||
// 19 files put it in the very-tiny tier (13,000-char envelope) while the
|
||||
// answer genuinely needs more, so the render loop spends its full allowed
|
||||
// overshoot — ~19.3K against a 19.5K ceiling. That leaves ~1% of headroom,
|
||||
// which is exactly why this is worth pinning: the bound that matters is the
|
||||
// host's ~25K inline cap, and above it the response is written to a file
|
||||
// the agent Reads back, undoing the point of the tool.
|
||||
const budget = getExploreOutputBudget(cg.getFiles().length);
|
||||
const hardCeiling = Math.min(Math.round(budget.maxOutputChars * 1.5), 25000);
|
||||
expect(response.length).toBeGreaterThan(budget.maxOutputChars);
|
||||
expect(response.length).toBeLessThanOrEqual(hardCeiling);
|
||||
expect(response.length).toBeLessThan(25000);
|
||||
});
|
||||
|
||||
it('records the shape of the allocation so a regression is legible', () => {
|
||||
// Not a gate — a snapshot of the split, so a future change that shifts the
|
||||
// numbers shows up in the diff rather than silently flipping a gate.
|
||||
const generated = generatedShare();
|
||||
const answer = answerShare();
|
||||
expect({
|
||||
generatedWinsEnvelope: generated > answer,
|
||||
workflowFileDelivers: (bytes.get('internal/usecase/payroll/cycle.go') ?? 0) > 0,
|
||||
builderFileDelivers: (bytes.get('internal/usecase/payroll/payslip_builder.go') ?? 0) > 0,
|
||||
}).toEqual({
|
||||
generatedWinsEnvelope: false,
|
||||
workflowFileDelivers: true,
|
||||
builderFileDelivers: true,
|
||||
});
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,975 @@
|
||||
/**
|
||||
* Score-proportional explore allocation, end to end (CG-14 / epic CG-1 / #1500).
|
||||
*
|
||||
* `explore-proportional-allocation.test.ts` pins `allocateExploreBudget` in
|
||||
* isolation; `explore-allocation-1500.test.ts` pins the reporter's Go shape.
|
||||
* What is left — and what this file owns — is everything the allocator only
|
||||
* *promises*: the render loop has to spend those reservations, the hard ceiling
|
||||
* has to catch the overshoot, and a degenerate or diffuse result set has to come
|
||||
* back usable rather than empty. Each of those is invisible to a unit test,
|
||||
* because the failure mode is not an exception — it is a response the agent
|
||||
* quietly abandons in favour of Read.
|
||||
*
|
||||
* Two halves:
|
||||
*
|
||||
* 1. **The self-query fixture's shape.** CG-6 declared a second regression
|
||||
* fixture beside payroll-go: this repo, asked "how does explore allocate its
|
||||
* output budget across files", spending 63% of its envelope on
|
||||
* `scripts/agent-eval/*.mjs` files that merely mention `explore` and
|
||||
* `BUDGET`, while `src/mcp/tools.ts` — the file that actually answers — sat
|
||||
* clipped at the flat `maxCharsPerFile`. That fixture reads THIS repo's live
|
||||
* index, so it belongs to the out-of-band probe
|
||||
* (`node scripts/agent-eval/probe-allocation.mjs self-query`) where its
|
||||
* numbers can move with the repo. Reproduced here as a synthetic project so
|
||||
* `npm test` owns the MECHANISM deterministically: a large relevant file, a
|
||||
* small genuinely-relevant helper, and an incidental name-collision script.
|
||||
*
|
||||
* 2. **Degenerate and diffuse result sets.** One file, no files, all files
|
||||
* scoring alike, a survey question. The proportional split divides by a total
|
||||
* weight and concentrates on a leader — both of which have a degenerate case
|
||||
* that ends in a division by zero or a starved response.
|
||||
*
|
||||
* Nothing here is platform-gated: fixtures are written through `path.join`, and
|
||||
* every path ASSERTED against is an indexed relative path, which extraction
|
||||
* normalizes to forward slashes on every platform (`normalizePath`, utils.ts).
|
||||
* A literal like `src/mcp/allocator.ts` is therefore correct on Windows too —
|
||||
* gate a new assertion with `it.runIf` only if it reaches for a real filesystem
|
||||
* path or a platform-specific separator.
|
||||
*/
|
||||
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import CodeGraph from '../src/index';
|
||||
import { ToolHandler, getExploreOutputBudget, EXPLORE_ALLOCATION } from '../src/mcp/tools';
|
||||
import { attributeSourceBytes } from '../src/mcp/explore-diagnostics';
|
||||
import type { ExploreDiagnosticReport } from '../src/mcp/explore-diagnostics';
|
||||
|
||||
/** The host's inline tool-result limit — above it the response is externalized. */
|
||||
const INLINE_CAP = 25000;
|
||||
|
||||
const DEBUG_ENV = 'CODEGRAPH_EXPLORE_DEBUG';
|
||||
|
||||
interface Project {
|
||||
dir: string;
|
||||
cg: CodeGraph;
|
||||
handler: ToolHandler;
|
||||
}
|
||||
|
||||
/** Build + index a throwaway project from a `{ relPath: source }` map. */
|
||||
async function buildProject(prefix: string, files: Record<string, string>): Promise<Project> {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), prefix));
|
||||
for (const [rel, body] of Object.entries(files)) {
|
||||
const abs = path.join(dir, rel);
|
||||
fs.mkdirSync(path.dirname(abs), { recursive: true });
|
||||
fs.writeFileSync(abs, body.trimStart());
|
||||
}
|
||||
const cg = CodeGraph.initSync(dir);
|
||||
await cg.indexAll();
|
||||
return { dir, cg, handler: new ToolHandler(cg) };
|
||||
}
|
||||
|
||||
function destroyProject(project?: Project): void {
|
||||
if (!project) return;
|
||||
project.cg.destroy();
|
||||
if (fs.existsSync(project.dir)) fs.rmSync(project.dir, { recursive: true, force: true });
|
||||
}
|
||||
|
||||
/**
|
||||
* One explore call, reduced to what the allocation assertions need — plus the
|
||||
* CG-4 per-file diagnostic, which is where the SCORE and the RESERVATION live.
|
||||
* The instrument is observational (byte-identical output either way), so reading
|
||||
* it here measures the same response the agent would have received.
|
||||
*/
|
||||
async function explore(project: Project, query: string) {
|
||||
// Outside the project root on purpose: a sidecar written INTO the indexed tree
|
||||
// is a new file the watcher can pick up mid-suite.
|
||||
const sidecar = path.join(fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-alloc-diag-')), 'report.jsonl');
|
||||
const previous = process.env[DEBUG_ENV];
|
||||
process.env[DEBUG_ENV] = sidecar;
|
||||
let result;
|
||||
try {
|
||||
result = await project.handler.execute('codegraph_explore', { query });
|
||||
} finally {
|
||||
if (previous === undefined) delete process.env[DEBUG_ENV];
|
||||
else process.env[DEBUG_ENV] = previous;
|
||||
}
|
||||
const text = result.content?.[0]?.text ?? '';
|
||||
const bytes = attributeSourceBytes(text);
|
||||
const lines = fs.existsSync(sidecar)
|
||||
? fs.readFileSync(sidecar, 'utf-8').trim().split('\n').filter(Boolean)
|
||||
: [];
|
||||
const report = JSON.parse(lines[lines.length - 1]!) as ExploreDiagnosticReport;
|
||||
fs.rmSync(path.dirname(sidecar), { recursive: true, force: true });
|
||||
const fileOf = (file: string) => report.files.find((f) => f.path === file);
|
||||
return {
|
||||
text,
|
||||
bytes,
|
||||
report,
|
||||
isError: result.isError === true,
|
||||
/** Relevance score the ranking pass gave this file. */
|
||||
score: (file: string) => fileOf(file)?.score ?? 0,
|
||||
/** Chars of source the allocator RESERVED for it, before anything rendered. */
|
||||
allowance: (file: string) => fileOf(file)?.allowance ?? 0,
|
||||
/** Which render path the loop took: `whole`, `clusters`, `focused`, `skeleton`. */
|
||||
render: (file: string) => fileOf(file)?.render ?? null,
|
||||
/**
|
||||
* Rank the ranking pass gave it (1 = the file the response leads with, and
|
||||
* the first the render loop reaches). Read off the record rather than from
|
||||
* the position in `report.files`, which the report re-sorts by delivered
|
||||
* bytes for legibility.
|
||||
*/
|
||||
rank: (file: string) => fileOf(file)?.rank ?? -1,
|
||||
/** Total source bytes delivered across every rendered file. */
|
||||
sourceTotal: () => [...bytes.values()].reduce((sum, n) => sum + n, 0),
|
||||
/** Fraction of the WHOLE response this file's source occupies. */
|
||||
share: (file: string) => (bytes.get(file) ?? 0) / (text.length || 1),
|
||||
shareUnder: (prefix: string) => {
|
||||
let total = 0;
|
||||
for (const [file, n] of bytes) if (file.startsWith(prefix)) total += n;
|
||||
return total / (text.length || 1);
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// ── 1. The self-query fixture's shape ───────────────────────────────────────
|
||||
|
||||
describe('#1500 fixture 2 — allocation followed FILE SIZE, not relevance', () => {
|
||||
/**
|
||||
* The three roles from the real fixture, at synthetic scale:
|
||||
*
|
||||
* - `src/mcp/allocator.ts` — stands in for `src/mcp/tools.ts`. Carries the
|
||||
* query's terms on real functions with real call edges, and is deliberately
|
||||
* too big to ship whole, so under the old rule it was clipped at the flat
|
||||
* `maxCharsPerFile` no matter how far it outscored its peers.
|
||||
* - `src/util/budget-math.ts` — stands in for `src/resolution/memory-budget.ts`.
|
||||
* Genuinely relevant (the allocator calls it) but scoring about half as
|
||||
* well — and small enough to ship WHOLE, which under the old rule was worth
|
||||
* more than being right.
|
||||
* - `scripts/eval-harness.mjs` — stands in for `scripts/agent-eval/*.mjs`. Its
|
||||
* only claim on the query is a file-scope `explore` and `BUDGET` that nothing
|
||||
* reads: the incidental collision CG-10 demoted.
|
||||
*
|
||||
* Measured on this fixture, reverting the render loop to the pre-CG-12 rules
|
||||
* (`fileBudget = maxCharsPerFile`, whole-file bound `maxCharsPerFile * 3`)
|
||||
* reproduces the report exactly — and every gate below goes red:
|
||||
*
|
||||
* | file | score | pre-CG-12 | CG-12 |
|
||||
* |-------------------------|-------|----------------|----------------|
|
||||
* | `src/mcp/allocator.ts` | 77.5 | 4,843 (39.7%) | 9,335 (80.1%) |
|
||||
* | `src/util/budget-math.ts` | 36.0 | 6,079 (49.8%) | 1,037 ( 8.9%) |
|
||||
*
|
||||
* The half-as-relevant file taking the larger share, purely on size, IS #1500.
|
||||
*/
|
||||
const QUERY = 'how does explore allocate its output budget across files';
|
||||
const ALLOCATOR = 'src/mcp/allocator.ts';
|
||||
const HELPER = 'src/util/budget-math.ts';
|
||||
const INCIDENTAL = 'scripts/eval-harness.mjs';
|
||||
|
||||
const allocatorPass = (index: number, name: string) => `
|
||||
/** ${name}: one pass of the explore output split. */
|
||||
export function ${name}(
|
||||
candidates: AllocationCandidate[],
|
||||
budget: ExploreOutputBudget,
|
||||
): Map<string, number> {
|
||||
const allowances = new Map<string, number>();
|
||||
const pool = clampOutputBudget(budget.maxOutputChars - ${index} * 200);
|
||||
const total = candidates.reduce((sum, candidate) => sum + candidate.score, 0);
|
||||
if (total <= 0) {
|
||||
return allowances;
|
||||
}
|
||||
const floors = Math.min(pool, 700 * candidates.length);
|
||||
const remainder = budgetRemainderAfterFloors(pool, floors);
|
||||
for (const candidate of candidates) {
|
||||
const floor = Math.floor(floors / candidates.length);
|
||||
const proportional = splitOutputEvenly(remainder, total, candidate.score);
|
||||
const boosted = candidate.spine ? proportional * 2 : proportional;
|
||||
const share = Math.min(floor + boosted, budget.maxCharsPerFile * 3);
|
||||
if (share <= 0) {
|
||||
continue;
|
||||
}
|
||||
allowances.set(candidate.path, share);
|
||||
}
|
||||
return allowances;
|
||||
}
|
||||
`;
|
||||
|
||||
/**
|
||||
* Neutral bulk for the helper file: real symbols that match NOTHING in the
|
||||
* query, so the file grows in BYTES without gaining relevance. That asymmetry
|
||||
* is the fixture — the real `memory-budget.ts` won 51% of the envelope against
|
||||
* a file scoring twice its score purely by being small enough to ship whole.
|
||||
*/
|
||||
const helperFiller = (n: number) => `
|
||||
export function normalizeLedgerRow${n}(row: string[], fallback: string): string[] {
|
||||
const trimmed = row.map((cell) => cell.trim()).filter((cell) => cell.length > 0);
|
||||
return trimmed.length > 0 ? trimmed : [fallback];
|
||||
}
|
||||
`;
|
||||
|
||||
const ALLOCATOR_SOURCE = `
|
||||
/** Explore budget allocation: splits the output envelope across relevant files. */
|
||||
export interface ExploreOutputBudget {
|
||||
maxOutputChars: number;
|
||||
maxCharsPerFile: number;
|
||||
defaultMaxFiles: number;
|
||||
}
|
||||
|
||||
export interface AllocationCandidate {
|
||||
path: string;
|
||||
score: number;
|
||||
spine: boolean;
|
||||
}
|
||||
${[
|
||||
'allocateExploreBudget',
|
||||
'reserveOutputPerFile',
|
||||
'distributeOutputBudget',
|
||||
'planExploreOutput',
|
||||
'spendExploreBudget',
|
||||
'balanceOutputAcrossFiles',
|
||||
'concentrateExploreOutput',
|
||||
'settleExploreAllocation',
|
||||
'apportionExploreBudget',
|
||||
'rationOutputAcrossFiles',
|
||||
'tallyExploreOutputBudget',
|
||||
'weighExploreAllocation',
|
||||
].map((name, i) => allocatorPass(i + 1, name)).join('')}
|
||||
import {
|
||||
clampOutputBudget,
|
||||
splitOutputEvenly,
|
||||
budgetRemainderAfterFloors,
|
||||
} from '../util/budget-math';
|
||||
`;
|
||||
|
||||
let project: Project;
|
||||
let run: Awaited<ReturnType<typeof explore>>;
|
||||
|
||||
beforeAll(async () => {
|
||||
project = await buildProject('codegraph-alloc-selfquery-', {
|
||||
[ALLOCATOR]: ALLOCATOR_SOURCE,
|
||||
[HELPER]: `
|
||||
/** Budget arithmetic the explore output allocator leans on. */
|
||||
export function clampOutputBudget(value: number): number {
|
||||
if (value < 0) return 0;
|
||||
return Math.floor(value);
|
||||
}
|
||||
|
||||
export function splitOutputEvenly(pool: number, total: number, score: number): number {
|
||||
if (total <= 0) return 0;
|
||||
return Math.floor((pool * score) / total);
|
||||
}
|
||||
|
||||
export function budgetRemainderAfterFloors(pool: number, floors: number): number {
|
||||
const remainder = pool - floors;
|
||||
return remainder > 0 ? remainder : 0;
|
||||
}
|
||||
|
||||
export function splitBudgetAcrossFiles(pool: number, fileCount: number): number {
|
||||
return fileCount > 0 ? Math.floor(pool / fileCount) : pool;
|
||||
}
|
||||
|
||||
export function describeOutputBudget(pool: number, perFile: number): string {
|
||||
return \`explore budget pool of \${pool} chars, \${perFile} per file\`;
|
||||
}
|
||||
${Array.from({ length: 22 }, (_, i) => helperFiller(i + 1)).join('')}`,
|
||||
[INCIDENTAL]: `
|
||||
// Eval harness. Mentions explore and BUDGET incidentally; nothing here allocates.
|
||||
const explore = 'explore';
|
||||
const BUDGET = 24000;
|
||||
|
||||
export function runHarness(repo) {
|
||||
const rows = [];
|
||||
for (const line of repo.split('\\n')) {
|
||||
rows.push(line.trim());
|
||||
}
|
||||
return rows;
|
||||
}
|
||||
|
||||
export function summarizeRun(rows) {
|
||||
return { count: rows.length, first: rows[0] };
|
||||
}
|
||||
`,
|
||||
'src/mcp/server.ts': `
|
||||
import { allocateExploreBudget } from './allocator';
|
||||
|
||||
export function serve(candidates: any[]) {
|
||||
return allocateExploreBudget(candidates, { maxOutputChars: 13000, maxCharsPerFile: 3800, defaultMaxFiles: 4 });
|
||||
}
|
||||
`,
|
||||
'src/util/logger.ts': `
|
||||
export function log(message: string): void {
|
||||
console.log(message);
|
||||
}
|
||||
`,
|
||||
});
|
||||
run = await explore(project, QUERY);
|
||||
}, 120_000);
|
||||
|
||||
afterAll(() => destroyProject(project));
|
||||
|
||||
describe('fixture shape', () => {
|
||||
it('indexes all three roles, so a zero share means demoted and not missing', () => {
|
||||
// Without this the incidental assertion below could pass vacuously — a file
|
||||
// that was never indexed also delivers 0 bytes.
|
||||
for (const rel of [ALLOCATOR, HELPER, INCIDENTAL]) {
|
||||
expect(project.cg.getFile(rel), `${rel} indexed`).toBeTruthy();
|
||||
}
|
||||
});
|
||||
|
||||
it('sizes the two files so the size-driven render split actually bites', () => {
|
||||
// The mechanism the epic is about. The answer file must be too big to ship
|
||||
// whole (so the old flat cap clipped it), and the helper small enough that
|
||||
// shipping it whole was always affordable under the old `maxCharsPerFile * 3`
|
||||
// bound. Without that asymmetry the fixture stops reproducing anything.
|
||||
const budget = getExploreOutputBudget(project.cg.getFiles().length);
|
||||
const answer = fs.readFileSync(path.join(project.dir, ALLOCATOR), 'utf-8');
|
||||
const helper = fs.readFileSync(path.join(project.dir, HELPER), 'utf-8');
|
||||
expect(answer.split('\n').length).toBeGreaterThan(280);
|
||||
expect(answer.length).toBeGreaterThan(budget.maxCharsPerFile * 3);
|
||||
expect(helper.split('\n').length).toBeLessThan(220);
|
||||
expect(helper.length).toBeLessThan(budget.maxCharsPerFile * 3);
|
||||
});
|
||||
|
||||
it('scores the answer file well above the helper it calls', () => {
|
||||
// The other half of the asymmetry: the reversal below only means something
|
||||
// if the file that used to WIN the envelope was the less relevant one.
|
||||
expect(run.score(ALLOCATOR)).toBeGreaterThan(run.score(HELPER) * 1.5);
|
||||
});
|
||||
});
|
||||
|
||||
describe('budget allocation', () => {
|
||||
it('gives the file that answers the question the majority of the envelope', () => {
|
||||
// The epic's acceptance bar for this fixture: >50%, from 18.5% at baseline.
|
||||
// Pre-CG-12 this file took 39.7% — behind the helper it calls.
|
||||
expect(run.share(ALLOCATOR)).toBeGreaterThan(0.5);
|
||||
});
|
||||
|
||||
it('lets the answer file spend multiples of the flat cap it used to be clipped at', () => {
|
||||
// The mechanism as a byte count rather than a share: this file is too big
|
||||
// to ship whole, so under the old rule its source was truncated at
|
||||
// `maxCharsPerFile` however far it outscored its peers. Its reservation is
|
||||
// now several times that cap. A build that re-imposes a flat per-file cap
|
||||
// fails HERE first — it delivered 4,843 against a 3,800 cap.
|
||||
const budget = getExploreOutputBudget(project.cg.getFiles().length);
|
||||
expect(run.bytes.get(ALLOCATOR) ?? 0).toBeGreaterThan(budget.maxCharsPerFile * 2);
|
||||
});
|
||||
|
||||
it('stops the smaller file winning on size — it no longer ships whole', () => {
|
||||
// The reversal, from the other side. The helper scores about half the
|
||||
// answer file and is small enough that the old whole-file bound shipped it
|
||||
// ENTIRE (6,079 chars, 49.8% of the envelope — more than the file that
|
||||
// answered the question). It now clusters inside its proportional share.
|
||||
const helperSource = fs.readFileSync(path.join(project.dir, HELPER), 'utf-8');
|
||||
const delivered = run.bytes.get(HELPER) ?? 0;
|
||||
expect(delivered).toBeGreaterThan(0);
|
||||
expect(delivered).toBeLessThan(helperSource.length);
|
||||
});
|
||||
|
||||
it('orders per-file shares by relevance, not by file size', () => {
|
||||
// Both files deliver — this is not concentration by elimination — but the
|
||||
// one that answers the question gets several times the bytes of the helper
|
||||
// it calls. Pre-CG-12 this ratio was 0.8, i.e. inverted.
|
||||
const answer = run.share(ALLOCATOR);
|
||||
const helper = run.share(HELPER);
|
||||
expect(helper).toBeGreaterThan(0);
|
||||
expect(answer).toBeGreaterThan(helper * 3);
|
||||
});
|
||||
|
||||
it('spends nothing on the incidental name collision', () => {
|
||||
expect(run.bytes.get(INCIDENTAL) ?? 0).toBe(0);
|
||||
expect(run.shareUnder('scripts/')).toBe(0);
|
||||
});
|
||||
|
||||
it('reserves in proportion to score, before anything renders', () => {
|
||||
// The reservations are the contract the render loop then spends. Asserting
|
||||
// them directly — not just the bytes that came out — separates "allocation
|
||||
// is proportional" from "the render loop happened to emit these sizes".
|
||||
const answerReserved = run.allowance(ALLOCATOR);
|
||||
const helperReserved = run.allowance(HELPER);
|
||||
expect(answerReserved).toBeGreaterThan(helperReserved);
|
||||
expect(answerReserved / helperReserved).toBeGreaterThan(run.score(ALLOCATOR) / run.score(HELPER) * 0.5);
|
||||
// Nothing is over-promised: the sum of reservations fits the pool, and the
|
||||
// pool fits the envelope. This is the invariant the whole epic rests on.
|
||||
expect(run.report.allocation.reserved).toBeLessThanOrEqual(run.report.allocation.pool);
|
||||
expect(run.report.allocation.pool).toBeLessThanOrEqual(run.report.budget.maxOutputChars);
|
||||
});
|
||||
|
||||
it('keeps the response inside the hard ceiling and under the inline cap', () => {
|
||||
// Two different bounds, and it matters which is which. `maxOutputChars`
|
||||
// bounds the RESERVATIONS (asserted above); the RESPONSE is bounded by
|
||||
// `hardCeiling` — 1.5x the envelope, capped at 25K — because the render
|
||||
// loop is allowed a bounded overshoot for the whole-file grace and an
|
||||
// oversize first cluster. The 25K is the one that must never move: past it
|
||||
// the host writes the result to a file the agent Reads back.
|
||||
const budget = getExploreOutputBudget(project.cg.getFiles().length);
|
||||
const hardCeiling = Math.min(Math.round(budget.maxOutputChars * 1.5), INLINE_CAP);
|
||||
expect(run.text.length).toBeLessThanOrEqual(hardCeiling);
|
||||
expect(run.text.length).toBeLessThan(INLINE_CAP);
|
||||
});
|
||||
|
||||
it('records the shape of the split so a regression is legible', () => {
|
||||
// Not a gate — a snapshot, so a change that shifts the split shows up in the
|
||||
// diff rather than silently flipping a threshold.
|
||||
expect({
|
||||
answerWinsEnvelope: run.share(ALLOCATOR) > run.share(HELPER),
|
||||
helperStillDelivers: (run.bytes.get(HELPER) ?? 0) > 0,
|
||||
incidentalDelivers: (run.bytes.get(INCIDENTAL) ?? 0) > 0,
|
||||
}).toEqual({
|
||||
answerWinsEnvelope: true,
|
||||
helperStillDelivers: true,
|
||||
incidentalDelivers: false,
|
||||
});
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
// ── 1b. CG-21: a reservation below the file's size must not lose its bytes ──
|
||||
|
||||
/**
|
||||
* The shape CG-15's agent A/B found in the wild, and the one thing the suite
|
||||
* above could not catch: a file whose reservation lands BELOW its own size.
|
||||
*
|
||||
* Express, `lib/utils.js` (5,293 B), the top-ranked file for
|
||||
* "res.send Content-Type ETag generateETag setETag":
|
||||
*
|
||||
* | | baseline | CG-12 |
|
||||
* |---|---|---|
|
||||
* | delivered | 6,380 (46.1%) whole | **583 (7.7%) cluster stub** |
|
||||
* | source envelope (13,000 budget) | 13,849 | **9,241** |
|
||||
*
|
||||
* It was reserved 3,870 and spent 583. The whole-file grace bound
|
||||
* (`allowance + min(800, allowance * 0.15)` = 4,450) sits just under the file,
|
||||
* so the whole-file render is declined; the fallback cluster render has three
|
||||
* matched symbols to work with and emits a stub. The other 3,287 chars were
|
||||
* neither delivered nor redistributed — **the pool shrank by a third against an
|
||||
* unchanged budget**, and the agent Read the file back four times.
|
||||
*
|
||||
* Everything about that is invisible to the fixtures above, and to the payroll
|
||||
* one: both SATURATE (`[over budget] [TRUNCATED]`, 23,599 of a 23,600 pool),
|
||||
* so there is no unspent reservation to lose. This fixture is built to sit in
|
||||
* the gap instead — a mid-sized top-ranked file with a THIN matched-symbol set,
|
||||
* sized just above its reservation — which is the combination that has to hold
|
||||
* for the defect to reproduce, and is why it shipped.
|
||||
*
|
||||
* The `fixture shape` block below is load-bearing, not scaffolding: every gate
|
||||
* here passes vacuously if the target ever drifts small enough for the grace
|
||||
* bound to cover it, so the window `0.6 × size <= reservation < size` is
|
||||
* asserted directly.
|
||||
*/
|
||||
describe('CG-21 — a reservation under the file size still buys the file', () => {
|
||||
// Names two symbols that live in ONE mid-sized file (the named-seed tier is
|
||||
// what puts it at rank 0) while the rest of the terms pull in its peers, so
|
||||
// the proportional split hands the target well under its own size.
|
||||
const QUERY = 'generateEtag compileEtag send response body';
|
||||
const TARGET = 'src/http/etag.ts';
|
||||
const RESPONSE = 'src/http/response.ts';
|
||||
const APPLICATION = 'src/http/application.ts';
|
||||
|
||||
/**
|
||||
* Bulk for the target: real, extractable symbols that match NOTHING in the
|
||||
* query. They make the file BIG without making it more relevant — which is
|
||||
* precisely how a file ends up reserved less than it is worth in bytes. Kept
|
||||
* dense (4 lines each) so the file stays well inside `WHOLE_FILE_MAX_LINES`
|
||||
* and the byte bound is the only thing that can decline the whole render.
|
||||
*/
|
||||
const inertFiller = (n: number) => `
|
||||
export function normalizeLedgerRow${n}(row: string[], fallback: string, separator: string): string[] {
|
||||
const trimmed = row.map((cell) => cell.trim()).filter((cell) => cell.length > 0 && cell !== separator);
|
||||
return trimmed.length > 0 ? trimmed : [fallback, separator, String(trimmed.length), 'ledger-row-${n}'];
|
||||
}
|
||||
`;
|
||||
|
||||
/**
|
||||
* The matched-symbol set, deliberately THIN and small. This is the second
|
||||
* half of the shape: with only these two tiny functions to cluster around,
|
||||
* the fallback render emits a few hundred chars and abandons the rest of the
|
||||
* reservation. A file with a fat matched set would spend its allowance the
|
||||
* ordinary way and never expose the bug.
|
||||
*/
|
||||
const TARGET_SOURCE = `
|
||||
/** ETag helpers. */
|
||||
export function generateEtag(body: string): string {
|
||||
return '"' + body.length.toString(16) + '"';
|
||||
}
|
||||
|
||||
export function compileEtag(setting: string): (body: string) => string {
|
||||
return setting === 'strong' ? generateEtag : (body: string) => 'W/' + generateEtag(body);
|
||||
}
|
||||
${Array.from({ length: 27 }, (_, i) => inertFiller(i + 1)).join('')}`;
|
||||
|
||||
const responseMethod = (name: string) => `
|
||||
public ${name}(body: string): string {
|
||||
const etag = compileEtag(this.etagSetting)(body);
|
||||
this.headers.set('etag', etag);
|
||||
return body;
|
||||
}
|
||||
`;
|
||||
|
||||
const RESPONSE_SOURCE = `
|
||||
import { compileEtag } from './etag';
|
||||
|
||||
/** The response object: sends a body and negotiates its representation. */
|
||||
export class ServerResponse {
|
||||
private headers = new Map<string, string>();
|
||||
private etagSetting = 'strong';
|
||||
${[
|
||||
'send',
|
||||
'sendBody',
|
||||
'sendResponse',
|
||||
'writeBody',
|
||||
'endResponse',
|
||||
'json',
|
||||
'setResponseBody',
|
||||
'flushResponseBody',
|
||||
].map(responseMethod).join('')}
|
||||
}
|
||||
`;
|
||||
|
||||
let project: Project;
|
||||
let run: Awaited<ReturnType<typeof explore>>;
|
||||
let targetSize = 0;
|
||||
|
||||
beforeAll(async () => {
|
||||
project = await buildProject('codegraph-alloc-cg21-', {
|
||||
[TARGET]: TARGET_SOURCE,
|
||||
[RESPONSE]: RESPONSE_SOURCE,
|
||||
[APPLICATION]: `
|
||||
import { ServerResponse } from './response';
|
||||
|
||||
/** The application: routes a request and hands the response its body. */
|
||||
export class Application {
|
||||
private routes = new Map<string, (res: ServerResponse) => string>();
|
||||
|
||||
public handleRequest(path: string, res: ServerResponse, body: string): string {
|
||||
const route = this.routes.get(path);
|
||||
return route ? route(res) : res.send(body);
|
||||
}
|
||||
|
||||
public registerResponseRoute(path: string, handler: (res: ServerResponse) => string): void {
|
||||
this.routes.set(path, handler);
|
||||
}
|
||||
}
|
||||
`,
|
||||
'src/http/request.ts': `
|
||||
/** The request object: carries the inbound body. */
|
||||
export class ServerRequest {
|
||||
public constructor(public readonly body: string) {}
|
||||
|
||||
public freshResponseBody(): string {
|
||||
return this.body.trim();
|
||||
}
|
||||
}
|
||||
`,
|
||||
'src/util/logger.ts': `
|
||||
export function log(message: string): void {
|
||||
console.log(message);
|
||||
}
|
||||
`,
|
||||
});
|
||||
targetSize = fs.readFileSync(path.join(project.dir, TARGET), 'utf-8').length;
|
||||
run = await explore(project, QUERY);
|
||||
}, 120_000);
|
||||
|
||||
afterAll(() => destroyProject(project));
|
||||
|
||||
describe('fixture shape', () => {
|
||||
it('ranks the target first, on a matched set of only two symbols', () => {
|
||||
// Rank 0 is what makes the loss expensive: this is the file the response
|
||||
// leads with, and the one the agent Reads back when it arrives as a stub.
|
||||
expect(run.rank(TARGET)).toBe(1);
|
||||
});
|
||||
|
||||
it('sizes the target ABOVE its reservation but inside the buy window', () => {
|
||||
// The whole assertion set below is vacuous outside this window, so it is
|
||||
// pinned here rather than assumed:
|
||||
// reservation >= size → the grace bound already covers it, and the
|
||||
// buy rule is never consulted (express's other
|
||||
// three queries look like this).
|
||||
// reservation < 0.6×size → the shortfall is real, clustering is the
|
||||
// right answer, and the carry-forward — not the
|
||||
// buy rule — is what conserves the bytes.
|
||||
const reserved = run.allowance(TARGET);
|
||||
expect(reserved).toBeGreaterThan(0);
|
||||
expect(reserved).toBeLessThan(targetSize);
|
||||
expect(reserved / targetSize).toBeGreaterThanOrEqual(EXPLORE_ALLOCATION.WHOLE_FILE_BUY_FRACTION);
|
||||
// ...and specifically OUTSIDE the grace bound, which is the pre-CG-21
|
||||
// rule. If grace alone could carry it, this fixture proves nothing.
|
||||
const graceBound = reserved + Math.min(
|
||||
EXPLORE_ALLOCATION.WHOLE_FILE_GRACE_MAX,
|
||||
Math.round(reserved * EXPLORE_ALLOCATION.WHOLE_FILE_GRACE_FRACTION),
|
||||
);
|
||||
expect(targetSize).toBeGreaterThan(graceBound);
|
||||
});
|
||||
|
||||
it('keeps the target inside the whole-file LINE bound, so only bytes can gate it', () => {
|
||||
// `WHOLE_FILE_MAX_LINES` (220 for a non-central file) is a separate gate
|
||||
// that also declines a whole render. If the fixture ever crossed it the
|
||||
// suite would go red for the wrong reason — and, worse, a genuine
|
||||
// regression in the BYTE bound would be masked by it.
|
||||
const lines = fs.readFileSync(path.join(project.dir, TARGET), 'utf-8').split('\n').length;
|
||||
expect(lines).toBeLessThanOrEqual(220);
|
||||
});
|
||||
});
|
||||
|
||||
describe('the reservation is spent', () => {
|
||||
it('delivers the target WHOLE rather than as a cluster stub', () => {
|
||||
// The headline. Pre-CG-21 this file rendered `clusters` and emitted a few
|
||||
// hundred chars against a multi-thousand-char reservation.
|
||||
expect(run.render(TARGET)).toBe('whole');
|
||||
});
|
||||
|
||||
it('spends more than the reservation, not a fraction of it', () => {
|
||||
// Stated as bytes so it bites independently of the render-mode label: a
|
||||
// build that renamed the whole path but still emitted a stub fails here.
|
||||
// Express: 583 delivered against 3,870 reserved.
|
||||
const delivered = run.bytes.get(TARGET) ?? 0;
|
||||
expect(delivered).toBeGreaterThanOrEqual(targetSize);
|
||||
expect(delivered).toBeGreaterThan(run.allowance(TARGET));
|
||||
});
|
||||
|
||||
it('leaves no rendered file both under its reservation and short of content', () => {
|
||||
// The defect stated as an invariant, which is what makes it general rather
|
||||
// than a re-assertion of the case above: a rendered file either SPENDS what
|
||||
// it was promised, or it ran out of file. Express's `lib/utils.js` did
|
||||
// neither — 583 delivered, 3,870 promised, 5,293 bytes of file sitting
|
||||
// there — and the difference was dropped rather than redistributed, which
|
||||
// is why the source envelope fell 13,849 → 9,241 on an unchanged budget.
|
||||
//
|
||||
// `response.ts` is the case the naive "spend the whole pool" version of
|
||||
// this test gets wrong: it delivers 1,635 of a 5,292 reservation and that
|
||||
// is CORRECT — the file is only 1,635 bytes. A pool cannot be spent past
|
||||
// the content that exists to fill it.
|
||||
for (const f of run.report.files) {
|
||||
if (!f.render || (f.emittedChars ?? 0) === 0) continue;
|
||||
const size = fs.readFileSync(path.join(project.dir, f.path), 'utf-8').length;
|
||||
expect(f.emittedChars, `${f.path} spent its reservation or ran out of file`)
|
||||
.toBeGreaterThanOrEqual(Math.min(f.allowance ?? 0, size));
|
||||
}
|
||||
});
|
||||
|
||||
it('holds the hard ceiling while doing it', () => {
|
||||
// The buy rule spends MORE than the reservation, so the bound that stops
|
||||
// it running away has to be re-proved here and not inherited: the
|
||||
// overshoot pool is finite, and the 25K inline cap is absolute — past it
|
||||
// the host writes the result to a file the agent Reads back.
|
||||
const budget = getExploreOutputBudget(project.cg.getFiles().length);
|
||||
const hardCeiling = Math.min(Math.round(budget.maxOutputChars * 1.5), INLINE_CAP);
|
||||
expect(run.text.length).toBeLessThanOrEqual(hardCeiling);
|
||||
expect(run.text.length).toBeLessThan(INLINE_CAP);
|
||||
});
|
||||
|
||||
it('still serves the peers — concentration, not a single-file response', () => {
|
||||
// The over-correction control for this fixture. Buying the target whole
|
||||
// must not eat the files below it: that is the trade the shared overshoot
|
||||
// pool refuses (it dropped `payslip_builder.go` when funding was per-file).
|
||||
const peers = [RESPONSE, APPLICATION].filter((f) => (run.bytes.get(f) ?? 0) > 0);
|
||||
expect(peers.length).toBeGreaterThan(0);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
/**
|
||||
* The other half of CG-21, and the half the whole-file buy rule cannot reach.
|
||||
*
|
||||
* Buying the file whole only helps when the reservation has already covered
|
||||
* most of it. Below that the shortfall is real — the file is several times its
|
||||
* reservation, and clustering IS the right render — but the bytes it cannot
|
||||
* spend still must not evaporate. Express, query "compileETag req.fresh":
|
||||
* `lib/utils.js` was reserved 3,809 and spent 791; the 3,018 chars it left had
|
||||
* to reach `lib/response.js` below it, which delivered 4,650 on a 1,895
|
||||
* reservation.
|
||||
*
|
||||
* So this fixture is deliberately the INVERSE of the one above: the leading
|
||||
* file is far too big for the buy rule to fire, and the assertion is on the
|
||||
* file BELOW it. Without this, `allowance = reserved` — the whole carry-forward
|
||||
* deleted — passes every other test in this file.
|
||||
*/
|
||||
describe('CG-21 — an unspendable reservation flows to the next file down', () => {
|
||||
// Names three tiny callables that all live in the SPRAWL file — the named-seed
|
||||
// tier is what puts a file with almost no matched content at rank 1 — plus one
|
||||
// term the absorber's methods carry, so it ranks second rather than cliffing.
|
||||
const QUERY = 'renderStaticScene renderInteractiveScene renderNewElementScene paintSceneLayer';
|
||||
// Rank 1: a huge file the query names two symbols in. Its reservation cannot
|
||||
// approach its size, so it clusters — and clusters thinly, because those two
|
||||
// symbols are all it matched.
|
||||
const SPRAWL = 'src/scene/sprawl.ts';
|
||||
// Rank 2: dense with matched symbols and bigger than any share it can be
|
||||
// reserved, so it will absorb whatever the file above it leaves.
|
||||
const ABSORBER = 'src/render/absorber.ts';
|
||||
|
||||
const inertBulk = (n: number) => `
|
||||
export function reconcileLedgerEntry${n}(rows: string[], fallback: string, separator: string): string[] {
|
||||
const trimmed = rows.map((cell) => cell.trim()).filter((cell) => cell.length > 0 && cell !== separator);
|
||||
return trimmed.length > 0 ? trimmed : [fallback, separator, String(trimmed.length), 'entry-${n}'];
|
||||
}
|
||||
`;
|
||||
|
||||
// Long ENOUGH, in lines, that the absorber cannot ship whole (220 lines is the
|
||||
// other whole-file gate). That matters: a file that renders whole ignores the
|
||||
// per-file budget entirely, and this fixture is about a budget being spent.
|
||||
const matchedPaint = (n: number) => `
|
||||
public paintSceneLayer${n}(canvas: string, scene: string, element: string): string {
|
||||
const appState = this.appState.get('layer${n}') ?? scene;
|
||||
const painted = canvas + '|' + appState + '|' + element;
|
||||
const stamped = painted + '|layer-${n}';
|
||||
const merged = stamped + '|' + scene + '|' + element;
|
||||
const settled = merged.split('|').filter((part) => part.length > 0).join('|');
|
||||
this.appState.set('layer${n}', settled);
|
||||
if (settled.length === 0) {
|
||||
return this.paint(scene, scene);
|
||||
}
|
||||
return this.paint(settled, scene);
|
||||
}
|
||||
`;
|
||||
|
||||
let project: Project;
|
||||
let run: Awaited<ReturnType<typeof explore>>;
|
||||
|
||||
beforeAll(async () => {
|
||||
project = await buildProject('codegraph-alloc-cg21-carry-', {
|
||||
[SPRAWL]: `
|
||||
import { Absorber } from '../render/absorber';
|
||||
|
||||
/** Scene sprawl: three one-line answers buried in a very large file. */
|
||||
export function renderStaticScene(scene: string): string {
|
||||
return new Absorber().paint(scene, scene);
|
||||
}
|
||||
|
||||
export function renderInteractiveScene(scene: string): string {
|
||||
return new Absorber().paint(scene, scene + ':interactive');
|
||||
}
|
||||
|
||||
export function renderNewElementScene(scene: string): string {
|
||||
return new Absorber().paint(scene, scene + ':new-element');
|
||||
}
|
||||
${Array.from({ length: 90 }, (_, i) => inertBulk(i + 1)).join('')}`,
|
||||
[ABSORBER]: `
|
||||
/** The renderer: many matched paint passes, all of them wanted. */
|
||||
export class Absorber {
|
||||
private appState = new Map<string, string>();
|
||||
|
||||
public paint(element: string, scene: string): string {
|
||||
return element + '|' + scene;
|
||||
}
|
||||
${Array.from({ length: 20 }, (_, i) => matchedPaint(i + 1)).join('')}
|
||||
}
|
||||
${/* Inert tail: pushes the absorber FAR past its reservation so the whole-file
|
||||
buy rule cannot fire on it either. Without this the absorber ships whole
|
||||
and the fixture measures the buy rule a second time instead of the
|
||||
carry-forward — which is exactly how it read on the first attempt. */
|
||||
Array.from({ length: 40 }, (_, i) => inertBulk(100 + i)).join('')}`,
|
||||
'src/util/logger.ts': `
|
||||
export function log(message: string): void {
|
||||
console.log(message);
|
||||
}
|
||||
`,
|
||||
});
|
||||
run = await explore(project, QUERY);
|
||||
}, 120_000);
|
||||
|
||||
afterAll(() => destroyProject(project));
|
||||
|
||||
it('leaves the leading file unable to spend its reservation', () => {
|
||||
// The precondition. If the sprawl file ever spends its share, there is no
|
||||
// slack, and the assertion below passes for no reason at all.
|
||||
const spent = run.bytes.get(SPRAWL) ?? 0;
|
||||
expect(spent).toBeGreaterThan(0);
|
||||
expect(spent).toBeLessThan(run.allowance(SPRAWL));
|
||||
// ...and it is out of reach of the buy rule, so this is genuinely the
|
||||
// carry-forward's case and not a second test of the fixture above.
|
||||
const size = fs.readFileSync(path.join(project.dir, SPRAWL), 'utf-8').length;
|
||||
expect(run.allowance(SPRAWL) / size).toBeLessThan(EXPLORE_ALLOCATION.WHOLE_FILE_BUY_FRACTION);
|
||||
});
|
||||
|
||||
it('hands the shortfall to the file below, which spends past its own reservation', () => {
|
||||
// The lever. Measured both ways on this fixture: with the carry-forward the
|
||||
// absorber delivers 9,297 against a 7,455 reservation; with
|
||||
// `allowance = reserved` it delivers 7,479 — its reservation and nothing
|
||||
// more, while the sprawl file's 4,408 unspent chars are dropped.
|
||||
//
|
||||
// The 1.1 margin is not padding. A cluster section can land a few chars over
|
||||
// the budget it was selected against (whole symbol ranges, never sliced
|
||||
// mid-method), so "delivered > reserved" alone is true by ~24 chars even on
|
||||
// the mutated build — a test that passes on the defect.
|
||||
const delivered = run.bytes.get(ABSORBER) ?? 0;
|
||||
expect(delivered).toBeGreaterThan(Math.round(run.allowance(ABSORBER) * 1.1));
|
||||
});
|
||||
|
||||
it('keeps the shortfall in the envelope instead of dropping it', () => {
|
||||
// The same lever read off the response as a whole, which is the form the
|
||||
// user actually feels: express's source envelope fell 13,849 → 9,241 on an
|
||||
// unchanged 13,000 budget because nothing picked up what `lib/utils.js`
|
||||
// could not spend. Here: 10,033 delivered with the carry-forward, 8,215
|
||||
// without.
|
||||
//
|
||||
// Stated against what a no-carry build could produce — the leader's actual
|
||||
// spend plus the absorber's own reservation — so it stays a statement about
|
||||
// the mechanism rather than a hard-coded byte count.
|
||||
const noCarryCeiling = (run.bytes.get(SPRAWL) ?? 0) + Math.round(run.allowance(ABSORBER) * 1.05);
|
||||
expect(run.sourceTotal()).toBeGreaterThan(noCarryCeiling);
|
||||
});
|
||||
|
||||
it('bounds the borrowing — slack concentrates, it does not consume', () => {
|
||||
// Carried slack is clamped to `MAX_SHARE` of the envelope, so an
|
||||
// under-spending leader cannot hand the file below it the whole response.
|
||||
// The bound is stated WITH the spine allowance (`SPINE_CEILING`, 1.5x)
|
||||
// folded in: a flow-path cluster is deliberately allowed past the per-file
|
||||
// share, and that predates CG-21 — writing the tighter bound here would
|
||||
// make this test fail on a build with no defect in it.
|
||||
const budget = getExploreOutputBudget(project.cg.getFiles().length);
|
||||
const clamp = Math.max(
|
||||
run.allowance(ABSORBER),
|
||||
Math.round(budget.maxOutputChars * EXPLORE_ALLOCATION.MAX_SHARE),
|
||||
);
|
||||
expect(run.bytes.get(ABSORBER) ?? 0).toBeLessThanOrEqual(Math.round(clamp * 1.5));
|
||||
// The anti-starvation half, and the one that would actually bite: the file
|
||||
// that lent the slack still gets rendered.
|
||||
expect(run.bytes.get(SPRAWL) ?? 0).toBeGreaterThan(0);
|
||||
expect(run.text.length).toBeLessThan(INLINE_CAP);
|
||||
});
|
||||
});
|
||||
|
||||
// ── 2. Degenerate and diffuse result sets ───────────────────────────────────
|
||||
|
||||
describe('allocation on degenerate result sets', () => {
|
||||
let project: Project;
|
||||
|
||||
beforeAll(async () => {
|
||||
// Four modules that are deliberate COPIES of each other, plus one unrelated
|
||||
// file. Copies are the pathological input for a proportional split: every
|
||||
// candidate carries the same weight, so the split divides by a denominator
|
||||
// that is entirely made of ties.
|
||||
const twin = (n: number) => `
|
||||
export class InventoryLedger${n} {
|
||||
private rows: number[] = [];
|
||||
|
||||
public recordInventoryMovement(quantity: number): void {
|
||||
this.rows.push(quantity);
|
||||
}
|
||||
|
||||
public settleInventoryLedger(): number {
|
||||
return this.rows.reduce((sum, row) => sum + row, 0);
|
||||
}
|
||||
}
|
||||
`;
|
||||
project = await buildProject('codegraph-alloc-degenerate-', {
|
||||
'src/ledger/one.ts': twin(1),
|
||||
'src/ledger/two.ts': twin(2),
|
||||
'src/ledger/three.ts': twin(3),
|
||||
'src/ledger/four.ts': twin(4),
|
||||
'src/unrelated/colors.ts': `
|
||||
export const PALETTE = ['oxblood', 'paper', 'ink'];
|
||||
|
||||
export function pickPaletteEntry(index: number): string {
|
||||
return PALETTE[index % PALETTE.length]!;
|
||||
}
|
||||
`,
|
||||
});
|
||||
}, 120_000);
|
||||
|
||||
afterAll(() => destroyProject(project));
|
||||
|
||||
it('does not starve anyone when every file scores identically', async () => {
|
||||
// The all-ties case, end to end: no division by zero, nobody cliffed for
|
||||
// being relatively weak (nothing IS relatively weak), and no single copy
|
||||
// sweeping the envelope on an arbitrary tiebreak.
|
||||
const run = await explore(project, 'how does the inventory ledger record and settle movements');
|
||||
expect(run.isError).toBe(false);
|
||||
const ledger = [...run.bytes].filter(([file]) => file.startsWith('src/ledger/'));
|
||||
expect(ledger.length).toBeGreaterThanOrEqual(2);
|
||||
const shares = ledger.map(([, n]) => n);
|
||||
expect(Math.max(...shares) / Math.min(...shares)).toBeLessThan(3);
|
||||
for (const [file, n] of ledger) {
|
||||
expect(n, `${file} starved`).toBeGreaterThan(0);
|
||||
}
|
||||
// The reservations behind those bytes divided cleanly too.
|
||||
expect(run.report.allocation.reserved).toBeLessThanOrEqual(run.report.allocation.pool);
|
||||
});
|
||||
|
||||
it('answers a single-file question without over-spending the envelope on it', async () => {
|
||||
const run = await explore(project, 'pickPaletteEntry');
|
||||
expect(run.isError).toBe(false);
|
||||
expect(run.bytes.get('src/unrelated/colors.ts') ?? 0).toBeGreaterThan(0);
|
||||
const budget = getExploreOutputBudget(project.cg.getFiles().length);
|
||||
// One dominant file still cannot exceed the share ceiling, and the response
|
||||
// as a whole still fits the envelope's hard ceiling.
|
||||
expect(run.bytes.get('src/unrelated/colors.ts')!)
|
||||
.toBeLessThanOrEqual(Math.round(budget.maxOutputChars * EXPLORE_ALLOCATION.MAX_SHARE));
|
||||
expect(run.text.length).toBeLessThan(INLINE_CAP);
|
||||
});
|
||||
|
||||
it('returns guidance rather than an error when nothing matches', async () => {
|
||||
// An `isError` response teaches the agent to abandon codegraph for the rest
|
||||
// of the session, so a zero-result allocation must stay success-shaped.
|
||||
const run = await explore(project, 'quantumFluxCapacitorHandshake');
|
||||
expect(run.isError).toBe(false);
|
||||
expect(run.text.length).toBeGreaterThan(0);
|
||||
expect(run.bytes.size).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('the diffuse-query control', () => {
|
||||
let project: Project;
|
||||
|
||||
beforeAll(async () => {
|
||||
// Six genuinely distinct subsystems, each a legitimate partial answer to a
|
||||
// survey question. Concentration is the epic's goal, but over-correcting here
|
||||
// costs a round-trip: the agent's fallback for an under-served survey is
|
||||
// Grep, not a second explore.
|
||||
const subsystem = (name: string, verb: string) => `
|
||||
export interface ${name}Options {
|
||||
retries: number;
|
||||
}
|
||||
|
||||
export class ${name}Service {
|
||||
constructor(private readonly options: ${name}Options) {}
|
||||
|
||||
public ${verb}Request(payload: string): string {
|
||||
return this.describe${name}() + ':' + payload;
|
||||
}
|
||||
|
||||
public describe${name}(): string {
|
||||
return '${name} with ' + this.options.retries + ' retries';
|
||||
}
|
||||
}
|
||||
`;
|
||||
project = await buildProject('codegraph-alloc-diffuse-', {
|
||||
'src/services/auth.ts': subsystem('Auth', 'authorize'),
|
||||
'src/services/billing.ts': subsystem('Billing', 'charge'),
|
||||
'src/services/search.ts': subsystem('Search', 'query'),
|
||||
'src/services/notify.ts': subsystem('Notify', 'publish'),
|
||||
'src/services/report.ts': subsystem('Report', 'render'),
|
||||
'src/services/audit.ts': subsystem('Audit', 'record'),
|
||||
});
|
||||
}, 120_000);
|
||||
|
||||
afterAll(() => destroyProject(project));
|
||||
|
||||
it('still returns a spread for a survey-style question', async () => {
|
||||
// The over-correction guard for CG-10's floor and CG-12's cliff together: a
|
||||
// question with no single right answer must come back as several usable
|
||||
// sections, not one file plus a pointer list.
|
||||
const run = await explore(project, 'what services does this project expose and what does each one do');
|
||||
expect(run.isError).toBe(false);
|
||||
const services = [...run.bytes].filter(([file]) => file.startsWith('src/services/'));
|
||||
expect(services.length).toBeGreaterThanOrEqual(3);
|
||||
const total = services.reduce((sum, [, n]) => sum + n, 0);
|
||||
expect(total).toBeGreaterThan(0);
|
||||
for (const [file, n] of services) {
|
||||
// Nobody is reduced to a fragment, and nobody swallows the response.
|
||||
expect(n, `${file} fragment`).toBeGreaterThan(200);
|
||||
expect(n / total, `${file} hogged the envelope`).toBeLessThan(0.8);
|
||||
}
|
||||
});
|
||||
|
||||
it('names whatever it could not show, so the spread stays completable', async () => {
|
||||
const run = await explore(project, 'what services does this project expose and what does each one do');
|
||||
const shown = [...run.bytes.keys()].filter((f) => f.startsWith('src/services/'));
|
||||
const missing = ['auth', 'billing', 'search', 'notify', 'report', 'audit']
|
||||
.map((n) => `src/services/${n}.ts`)
|
||||
.filter((f) => !shown.includes(f));
|
||||
for (const file of missing) {
|
||||
expect(run.text, `${file} dropped without a pointer`).toContain(file);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -40,6 +40,28 @@ describe('codegraph_explore — blast radius', () => {
|
||||
path.join(src, 'leaf.ts'),
|
||||
`export function lonelyLeaf() { return 42; }\n`,
|
||||
);
|
||||
// `deepHelper` is only called by production code (`midCaller`), but the
|
||||
// test file exercises it transitively — 2 caller hops up (#1475).
|
||||
fs.writeFileSync(
|
||||
path.join(src, 'util.ts'),
|
||||
`export function deepHelper() { return 1; }\n`,
|
||||
);
|
||||
fs.writeFileSync(
|
||||
path.join(src, 'mid.ts'),
|
||||
`import { deepHelper } from './util';\n` +
|
||||
`export function midCaller() { return deepHelper(); }\n`,
|
||||
);
|
||||
fs.writeFileSync(
|
||||
path.join(src, 'mid.test.ts'),
|
||||
`import { midCaller } from './mid';\n` +
|
||||
`export function checkMid() { return midCaller(); }\n`,
|
||||
);
|
||||
// `untestedHelper` has a caller but no test anywhere up its caller chain.
|
||||
fs.writeFileSync(
|
||||
path.join(src, 'untested.ts'),
|
||||
`export function untestedHelper() { return 3; }\n` +
|
||||
`export function untestedCaller() { return untestedHelper(); }\n`,
|
||||
);
|
||||
|
||||
cg = CodeGraph.initSync(testDir, { config: { include: ['**/*.ts'], exclude: [] } });
|
||||
await cg.indexAll();
|
||||
@@ -60,8 +82,28 @@ describe('codegraph_explore — blast radius', () => {
|
||||
expect(text).toMatch(/caller/); // a caller count is reported
|
||||
// It names WHERE (the caller file) — not the caller's source body.
|
||||
expect(text).toContain('feature.ts');
|
||||
// Test coverage is surfaced (either the covering test file, or the warning).
|
||||
expect(text).toMatch(/tests:.*feature\.test\.ts|no covering tests/);
|
||||
// The direct covering test file is surfaced.
|
||||
expect(text).toMatch(/tests:.*feature\.test\.ts/);
|
||||
});
|
||||
|
||||
it('surfaces tests that cover a symbol transitively through its callers (#1475)', async () => {
|
||||
const res = await handler.execute('codegraph_explore', { query: 'deepHelper' });
|
||||
const text = res.content[0].text;
|
||||
|
||||
// deepHelper's only direct caller is production code, but mid.test.ts sits
|
||||
// one more hop up — that must NOT read as "no tests".
|
||||
expect(text).toMatch(/`deepHelper`[^\n]*tested via callers:[^\n]*mid\.test\.ts/);
|
||||
const line = text.split('\n').find((l: string) => l.startsWith('- `deepHelper`'));
|
||||
expect(line).not.toMatch(/no tests found|no covering tests/);
|
||||
});
|
||||
|
||||
it('states only what was measured when no test exists up the caller chain', async () => {
|
||||
const res = await handler.execute('codegraph_explore', { query: 'untestedHelper' });
|
||||
const text = res.content[0].text;
|
||||
|
||||
// Bounded claim, no warning glyph — the tool verified nothing beyond 3 hops.
|
||||
expect(text).toMatch(/`untestedHelper`[^\n]*no tests found within 3 caller hops/);
|
||||
expect(text).not.toContain('⚠️ no covering tests found');
|
||||
});
|
||||
|
||||
it('omits symbols that have no dependents from the blast radius', async () => {
|
||||
|
||||
@@ -0,0 +1,170 @@
|
||||
/**
|
||||
* Regression gate for CLUSTER-LEVEL STARVATION inside one file (task CG-36).
|
||||
*
|
||||
* A file's ranked clusters used to be all-or-nothing past the first one: the
|
||||
* top-ranked cluster was taken (shrunk to fit if it had to be), and every
|
||||
* cluster below it was rendered whole and then either fit the remainder or was
|
||||
* dropped entirely. On a file whose top-ranked cluster is TRIVIAL that discards
|
||||
* the answer — django's `db/models/sql/query.py` kept a 22-line glue cluster and
|
||||
* dropped the 624-line `Query` body beneath it, spending 1,923 of a 7,947
|
||||
* reservation, and okhttp's `RealInterceptorChain.kt` did the same behind its
|
||||
* import header.
|
||||
*
|
||||
* What makes it hard to see is that the response stays FULL: the unspent
|
||||
* reservation carries forward exactly as designed, so a lower-scoring file takes
|
||||
* the bytes and every envelope-share measure still looks healthy. The gate is
|
||||
* therefore per-file spend, not share.
|
||||
*
|
||||
* Two fixtures, pulling in opposite directions — read them together:
|
||||
*
|
||||
* - `starved-cluster-ts` is the defect. Its answer-bearing cluster must be
|
||||
* SHRUNK into whatever the trivial cluster left, not dropped.
|
||||
* - `dense-header-ts` is the Session.swift shape that cluster ranking puts
|
||||
* importance ahead of density FOR. Its query's methods sit ~200 lines under
|
||||
* a dense property list, and they must keep winning the budget. Any future
|
||||
* rework of selection or shrinking has to satisfy both.
|
||||
*/
|
||||
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import CodeGraph from '../src/index';
|
||||
import { ToolHandler } from '../src/mcp/tools';
|
||||
import type { ExploreDiagnosticReport } from '../src/mcp/explore-diagnostics';
|
||||
|
||||
interface Run {
|
||||
dir: string;
|
||||
cg: CodeGraph;
|
||||
response: string;
|
||||
report: ExploreDiagnosticReport;
|
||||
}
|
||||
|
||||
/** Copy a fixture tree to a temp dir, index it, and run one explore call. */
|
||||
async function runFixture(fixture: string, query: string): Promise<Run> {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cg36-'));
|
||||
fs.cpSync(path.join(__dirname, 'fixtures', fixture), dir, { recursive: true });
|
||||
fs.rmSync(path.join(dir, '.codegraph'), { recursive: true, force: true });
|
||||
|
||||
const cg = CodeGraph.initSync(dir);
|
||||
await cg.indexAll();
|
||||
|
||||
const sidecar = path.join(dir, 'explore-diag.jsonl');
|
||||
const previous = process.env.CODEGRAPH_EXPLORE_DEBUG;
|
||||
process.env.CODEGRAPH_EXPLORE_DEBUG = sidecar;
|
||||
let response: string;
|
||||
try {
|
||||
response = (await new ToolHandler(cg).execute('codegraph_explore', { query }))
|
||||
.content?.[0]?.text ?? '';
|
||||
} finally {
|
||||
if (previous === undefined) delete process.env.CODEGRAPH_EXPLORE_DEBUG;
|
||||
else process.env.CODEGRAPH_EXPLORE_DEBUG = previous;
|
||||
}
|
||||
const written = fs.readFileSync(sidecar, 'utf-8').trim().split('\n').filter(Boolean);
|
||||
return { dir, cg, response, report: JSON.parse(written[written.length - 1]!) };
|
||||
}
|
||||
|
||||
function teardown(run: Run | undefined): void {
|
||||
if (!run) return;
|
||||
run.cg.destroy();
|
||||
if (fs.existsSync(run.dir)) fs.rmSync(run.dir, { recursive: true, force: true });
|
||||
}
|
||||
|
||||
describe('CG-36 — a trivial cluster must not starve the answer-bearing one', () => {
|
||||
const TARGET = 'src/pipeline/chain.ts';
|
||||
const QUERY = 'how does a request travel from sendRequest to the socket';
|
||||
let run: Run;
|
||||
let target: ExploreDiagnosticReport['files'][number];
|
||||
|
||||
beforeAll(async () => {
|
||||
run = await runFixture('starved-cluster-ts', QUERY);
|
||||
target = run.report.files.find((f) => f.path === TARGET)!;
|
||||
}, 120_000);
|
||||
|
||||
afterAll(() => teardown(run));
|
||||
|
||||
describe('fixture shape — if this rots, the gate below means nothing', () => {
|
||||
it('renders through the cluster path, with the answer past the trivial helper', () => {
|
||||
expect(target, `${TARGET} is not among the ranked candidates`).toBeDefined();
|
||||
expect(target.render).toBe('clusters');
|
||||
// The helper the entry point calls directly, and the class it does not.
|
||||
const nodes = run.cg.getNodesInFile(TARGET);
|
||||
const helper = nodes.find((n) => n.name === 'describeChain')!;
|
||||
const proceed = nodes.find((n) => n.name === 'proceed')!;
|
||||
expect(helper).toBeDefined();
|
||||
expect(proceed).toBeDefined();
|
||||
// Far enough apart to cluster separately at any gap threshold we ship.
|
||||
expect(proceed.startLine - helper.endLine).toBeGreaterThan(20);
|
||||
});
|
||||
|
||||
it('reserves the file the largest share, so an unspent share is a defect', () => {
|
||||
expect(target.allowance ?? 0).toBeGreaterThan(4000);
|
||||
const others = run.report.files.filter((f) => f.path !== TARGET);
|
||||
for (const f of others) expect(f.allowance ?? 0).toBeLessThan(target.allowance!);
|
||||
});
|
||||
});
|
||||
|
||||
describe('the gate', () => {
|
||||
it('spends most of the reservation it was given', () => {
|
||||
// 28.8% on the CG-24 epic tip, 131% (its reservation plus carry-forward
|
||||
// slack it can now actually use) with the fix. The bar is deliberately
|
||||
// well below both so ordinary budget movement does not fail the suite.
|
||||
expect(target.finalChars / target.allowance!).toBeGreaterThan(0.6);
|
||||
});
|
||||
|
||||
it('delivers the flow the query asked about, not just the helper beside it', () => {
|
||||
// Both ends of the in-file flow, in the cluster that used to be dropped.
|
||||
expect(run.response).toContain('async proceed(request: PipelineRequest)');
|
||||
expect(run.response).toContain('private async writeAndRead(request: PipelineRequest)');
|
||||
});
|
||||
|
||||
it('keeps the response inside the hard ceiling', () => {
|
||||
expect(run.report.envelope.chars).toBeLessThanOrEqual(run.report.budget.hardCeiling);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('CG-36 — a dense declaration block must not bury the query\'s methods', () => {
|
||||
const TARGET = 'src/net/session.ts';
|
||||
const QUERY = 'how does perform create a URLRequest and start the task';
|
||||
let run: Run;
|
||||
let target: ExploreDiagnosticReport['files'][number];
|
||||
|
||||
beforeAll(async () => {
|
||||
run = await runFixture('dense-header-ts', QUERY);
|
||||
target = run.report.files.find((f) => f.path === TARGET)!;
|
||||
}, 120_000);
|
||||
|
||||
afterAll(() => teardown(run));
|
||||
|
||||
describe('fixture shape — if this rots, the gate below means nothing', () => {
|
||||
it('has a dense low-importance header and the named methods far below it', () => {
|
||||
expect(target, `${TARGET} is not among the ranked candidates`).toBeDefined();
|
||||
expect(target.render).toBe('clusters');
|
||||
const nodes = run.cg.getNodesInFile(TARGET);
|
||||
const perform = nodes.find((n) => n.name === 'perform')!;
|
||||
expect(perform).toBeDefined();
|
||||
// The header block: many adjacent declarations above the first named
|
||||
// method, which is what makes it the densest region of the file.
|
||||
const above = nodes.filter((n) => n.endLine < perform.startLine
|
||||
&& (n.kind === 'property' || n.kind === 'field' || n.kind === 'method'));
|
||||
expect(above.length).toBeGreaterThan(20);
|
||||
expect(perform.startLine).toBeGreaterThan(150);
|
||||
});
|
||||
});
|
||||
|
||||
describe('the gate', () => {
|
||||
it('delivers all three methods the query named', () => {
|
||||
expect(run.response).toContain('async perform(url: string, method: string');
|
||||
expect(run.response).toContain('didCreateURLRequest(request: URLRequest)');
|
||||
expect(run.response).toContain('task(request: URLRequest, identifier: number)');
|
||||
});
|
||||
|
||||
it('spends the file\'s reservation on them', () => {
|
||||
expect(target.finalChars / target.allowance!).toBeGreaterThan(0.6);
|
||||
});
|
||||
|
||||
it('keeps the response inside the hard ceiling', () => {
|
||||
expect(run.report.envelope.chars).toBeLessThanOrEqual(run.report.budget.hardCeiling);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,360 @@
|
||||
/**
|
||||
* Cross-call source dedup (CG-18).
|
||||
*
|
||||
* A later `codegraph_explore` call in a session must not re-send source an
|
||||
* earlier call already delivered — but every byte it withholds has to be
|
||||
* replaced by a POINTER, never a silence. That asymmetry is what this suite
|
||||
* guards, because the two failure directions cost wildly different amounts: a
|
||||
* duplicate range wastes a few thousand chars, while a response that reads as
|
||||
* "codegraph doesn't have it" costs a Read — and one or two of those early in a
|
||||
* session teach an agent to stop calling the tool at all.
|
||||
*
|
||||
* Three layers:
|
||||
* 1. the range algebra — what is withheld, and the thresholds that stop it
|
||||
* from shredding a block into slivers;
|
||||
* 2. the fingerprint gate — an edit between two calls must re-emit, since a
|
||||
* pointer to pre-edit source is worse than no dedup at all;
|
||||
* 3. the handler seam — a real second call against a real index: no duplicate
|
||||
* ranges, a pointer for everything withheld, the reclaimed budget spent on
|
||||
* source the agent has NOT seen, and never an all-pointer response.
|
||||
*/
|
||||
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import CodeGraph from '../src/index';
|
||||
import { ToolHandler } from '../src/mcp/tools';
|
||||
import { ExploreSessionState, type ExploreProjectState } from '../src/mcp/explore-session-state';
|
||||
import {
|
||||
EXPLORE_DEDUP,
|
||||
dedupeRange,
|
||||
fileFingerprint,
|
||||
formatBackReference,
|
||||
intersectRange,
|
||||
mergeRanges,
|
||||
servedRangesForFile,
|
||||
subtractRange,
|
||||
symbolsInSpans,
|
||||
} from '../src/mcp/explore-dedup';
|
||||
|
||||
const FIXTURE_SRC = path.join(__dirname, 'fixtures', 'payroll-go');
|
||||
const QUERY = 'how does payroll cycle create and calculate payslips?';
|
||||
const POINTER = 'Already sent earlier in this conversation';
|
||||
|
||||
/** A prior-state shaped like the session tracker's, for the algebra tests. */
|
||||
function prior(files: Array<{ path: string; ranges: Array<[number, number]>; fingerprint?: string }>): ExploreProjectState {
|
||||
return {
|
||||
projectRoot: '/repo',
|
||||
callCount: 1,
|
||||
responseBytes: 1000,
|
||||
calls: [{
|
||||
index: 1,
|
||||
projectRoot: '/repo',
|
||||
query: 'q',
|
||||
sourceBytes: 500,
|
||||
responseBytes: 1000,
|
||||
files: files.map((f) => ({
|
||||
path: f.path,
|
||||
ranges: f.ranges.map(([start, end]) => ({ start, end })),
|
||||
bytes: 500,
|
||||
fingerprint: f.fingerprint,
|
||||
})),
|
||||
}],
|
||||
};
|
||||
}
|
||||
|
||||
describe('range algebra', () => {
|
||||
it('subtracts a held span out of the middle of an intended one', () => {
|
||||
expect(subtractRange({ start: 1, end: 100 }, [{ start: 20, end: 40 }]))
|
||||
.toEqual([{ start: 1, end: 19 }, { start: 41, end: 100 }]);
|
||||
});
|
||||
|
||||
it('subtracts held spans at either edge, and a full cover to nothing', () => {
|
||||
expect(subtractRange({ start: 10, end: 50 }, [{ start: 1, end: 20 }]))
|
||||
.toEqual([{ start: 21, end: 50 }]);
|
||||
expect(subtractRange({ start: 10, end: 50 }, [{ start: 30, end: 90 }]))
|
||||
.toEqual([{ start: 10, end: 29 }]);
|
||||
expect(subtractRange({ start: 10, end: 50 }, [{ start: 1, end: 90 }])).toEqual([]);
|
||||
});
|
||||
|
||||
it('intersects to exactly what both sides hold', () => {
|
||||
expect(intersectRange({ start: 10, end: 50 }, [{ start: 1, end: 20 }, { start: 45, end: 80 }]))
|
||||
.toEqual([{ start: 10, end: 20 }, { start: 45, end: 50 }]);
|
||||
expect(intersectRange({ start: 10, end: 50 }, [{ start: 60, end: 80 }])).toEqual([]);
|
||||
});
|
||||
|
||||
it('merges touching spans — two adjacent blocks are one block of source', () => {
|
||||
expect(mergeRanges([{ start: 5, end: 9 }, { start: 10, end: 12 }, { start: 40, end: 41 }]))
|
||||
.toEqual([{ start: 5, end: 12 }, { start: 40, end: 41 }]);
|
||||
});
|
||||
|
||||
it('emits ONLY the delta when a later call wants a wider window', () => {
|
||||
// Call 1 sent the method; call 2 wants the class around it.
|
||||
const { emit, covered } = dedupeRange({ start: 80, end: 200 }, [{ start: 100, end: 140 }]);
|
||||
expect(covered).toEqual([{ start: 100, end: 140 }]);
|
||||
expect(emit).toEqual([{ start: 80, end: 99 }, { start: 141, end: 200 }]);
|
||||
});
|
||||
|
||||
it('withholds nothing when the overlap is smaller than a chunk worth pointing at', () => {
|
||||
// Context padding and signature lines land here. Replacing them costs more
|
||||
// in pointer text than the source is worth, and shreds the block.
|
||||
const overlap = EXPLORE_DEDUP.MIN_COVERED_LINES - 1;
|
||||
const { emit, covered } = dedupeRange({ start: 1, end: 100 }, [{ start: 10, end: 10 + overlap - 1 }]);
|
||||
expect(covered).toEqual([]);
|
||||
expect(emit).toEqual([{ start: 1, end: 100 }]);
|
||||
});
|
||||
|
||||
it('leaves an untouched span exactly as it was', () => {
|
||||
expect(dedupeRange({ start: 1, end: 50 }, [{ start: 200, end: 400 }]))
|
||||
.toEqual({ emit: [{ start: 1, end: 50 }], covered: [] });
|
||||
expect(dedupeRange({ start: 1, end: 50 }, []))
|
||||
.toEqual({ emit: [{ start: 1, end: 50 }], covered: [] });
|
||||
});
|
||||
});
|
||||
|
||||
describe('the fingerprint gate', () => {
|
||||
const FP = fileFingerprint('package main\nfunc main() {}\n');
|
||||
|
||||
it('returns the spans a call served for a file whose bytes are unchanged', () => {
|
||||
const state = prior([{ path: 'a.go', ranges: [[1, 40], [60, 80]], fingerprint: FP }]);
|
||||
expect(servedRangesForFile(state, 'a.go', FP)).toEqual([{ start: 1, end: 40 }, { start: 60, end: 80 }]);
|
||||
});
|
||||
|
||||
it('returns NOTHING once the file has been edited — a pointer would be wrong', () => {
|
||||
const state = prior([{ path: 'a.go', ranges: [[1, 40]], fingerprint: FP }]);
|
||||
const edited = fileFingerprint('package main\nfunc main() { changed() }\n');
|
||||
expect(servedRangesForFile(state, 'a.go', edited)).toEqual([]);
|
||||
});
|
||||
|
||||
it('ignores a record that cannot prove what it served', () => {
|
||||
const state = prior([{ path: 'a.go', ranges: [[1, 40]] }]);
|
||||
expect(servedRangesForFile(state, 'a.go', FP)).toEqual([]);
|
||||
});
|
||||
|
||||
it('never crosses files, and is empty for an untracked session', () => {
|
||||
const state = prior([{ path: 'a.go', ranges: [[1, 40]], fingerprint: FP }]);
|
||||
expect(servedRangesForFile(state, 'b.go', FP)).toEqual([]);
|
||||
expect(servedRangesForFile(null, 'a.go', FP)).toEqual([]);
|
||||
});
|
||||
|
||||
it('distinguishes two files that hash the same prefix but differ in length', () => {
|
||||
expect(fileFingerprint('abc')).not.toBe(fileFingerprint('abcd'));
|
||||
expect(fileFingerprint('abc')).toBe(fileFingerprint('abc'));
|
||||
});
|
||||
});
|
||||
|
||||
describe('the back-reference itself', () => {
|
||||
const covered = [{ start: 100, end: 240 }];
|
||||
|
||||
it('names the file, the span and the symbols, and says the copy is still good', () => {
|
||||
const text = formatBackReference('internal/x.go', covered, ['RunCycle', 'BuildPayslip'], { partial: false });
|
||||
expect(text).toContain('internal/x.go');
|
||||
expect(text).toContain('L100-240');
|
||||
expect(text).toContain('RunCycle, BuildPayslip');
|
||||
expect(text).toContain(POINTER);
|
||||
expect(text).toContain('unchanged on disk');
|
||||
});
|
||||
|
||||
it('never tells the agent to Read, in either shape', () => {
|
||||
for (const partial of [true, false]) {
|
||||
const text = formatBackReference('x.go', covered, ['A'], { partial });
|
||||
expect(text).toMatch(/do NOT Read/i);
|
||||
expect(text).not.toMatch(/\bRead (this|the) file (for|to)\b/i);
|
||||
expect(text).not.toMatch(/omitted|unavailable|could not/i);
|
||||
}
|
||||
});
|
||||
|
||||
it('says the block below is only the NEW lines when the call still sends some', () => {
|
||||
expect(formatBackReference('x.go', covered, [], { partial: true })).toContain('NEW lines');
|
||||
expect(formatBackReference('x.go', covered, [], { partial: false })).toContain('not repeated here');
|
||||
});
|
||||
|
||||
it('names only symbols that actually fall in the withheld spans', () => {
|
||||
const nodes = [
|
||||
{ name: 'InSpan', kind: 'function', startLine: 110, endLine: 130 },
|
||||
{ name: 'Outside', kind: 'function', startLine: 300, endLine: 320 },
|
||||
{ name: 'AnImport', kind: 'import', startLine: 105, endLine: 105 },
|
||||
];
|
||||
expect(symbolsInSpans(nodes, covered)).toEqual(['InSpan']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('a second call against a real index', () => {
|
||||
let testDir: string;
|
||||
let cg: CodeGraph;
|
||||
let handler: ToolHandler;
|
||||
|
||||
beforeAll(async () => {
|
||||
testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cg18-'));
|
||||
fs.cpSync(FIXTURE_SRC, testDir, { recursive: true });
|
||||
fs.rmSync(path.join(testDir, '.codegraph'), { recursive: true, force: true });
|
||||
cg = CodeGraph.initSync(testDir);
|
||||
await cg.indexAll();
|
||||
handler = new ToolHandler(cg);
|
||||
}, 120_000);
|
||||
|
||||
afterAll(() => {
|
||||
if (cg) cg.destroy();
|
||||
if (testDir && fs.existsSync(testDir)) fs.rmSync(testDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
const explore = (query: string, session?: ExploreSessionState, args: Record<string, unknown> = {}) =>
|
||||
handler.execute('codegraph_explore', { query, ...args }, session).then((r) => r.content[0]!.text);
|
||||
|
||||
/**
|
||||
* The line numbers actually inside each file's fenced source. Read off the
|
||||
* RESPONSE, not the bookkeeping — "no duplicate ranges" is a claim about what
|
||||
* the agent received, and checking it against the record we also wrote would
|
||||
* prove only that the two agree.
|
||||
*/
|
||||
function fencedLines(text: string): Map<string, Set<number>> {
|
||||
const out = new Map<string, Set<number>>();
|
||||
let current: string | null = null;
|
||||
let inFence = false;
|
||||
for (const line of text.split('\n')) {
|
||||
const header = /^\*\*`([^`]+)`\*\*/.exec(line);
|
||||
if (header && !inFence) { current = header[1]!; continue; }
|
||||
if (!inFence && current && line.startsWith('```')) { inFence = true; continue; }
|
||||
if (inFence && line === '```') { inFence = false; continue; }
|
||||
if (!inFence || !current) continue;
|
||||
const numbered = /^(\d+)\t/.exec(line);
|
||||
if (!numbered) continue;
|
||||
if (!out.has(current)) out.set(current, new Set());
|
||||
out.get(current)!.add(Number(numbered[1]));
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
it('never re-sends a line it already sent, and points at every line it withholds', async () => {
|
||||
const session = new ExploreSessionState();
|
||||
const first = await explore(QUERY, session);
|
||||
const second = await explore(QUERY, session);
|
||||
|
||||
const before = fencedLines(first);
|
||||
const after = fencedLines(second);
|
||||
expect(after.size).toBeGreaterThan(0);
|
||||
|
||||
// Every file whose source the second call withheld carries a pointer, and
|
||||
// the pointer names it.
|
||||
expect(second).toContain(POINTER);
|
||||
for (const [file, lines] of before) {
|
||||
const repeated = [...(after.get(file) ?? [])].filter((n) => lines.has(n));
|
||||
if (repeated.length === 0) continue;
|
||||
// The only sanctioned repeat is the anti-abandonment restore, which fires
|
||||
// ONLY when the call found nothing new to say — and this one did.
|
||||
throw new Error(`call 2 re-sent ${file} lines ${repeated.slice(0, 5).join(',')}`);
|
||||
}
|
||||
}, 120_000);
|
||||
|
||||
it('spends the reclaimed bytes on source the agent has not seen', async () => {
|
||||
const session = new ExploreSessionState();
|
||||
const first = await explore(QUERY, session);
|
||||
const second = await explore(QUERY, session);
|
||||
|
||||
const before = fencedLines(first);
|
||||
const after = fencedLines(second);
|
||||
const fresh = [...after.entries()].reduce(
|
||||
(sum, [file, lines]) => sum + [...lines].filter((n) => !(before.get(file)?.has(n))).length, 0);
|
||||
// Not merely "smaller": a shrunken response is what dedup must NOT produce.
|
||||
// The freed budget has to come back as lines the first call never sent.
|
||||
expect(fresh).toBeGreaterThan(20);
|
||||
expect(second.length).toBeLessThan(first.length);
|
||||
}, 120_000);
|
||||
|
||||
it('re-emits in full when the file changed between the two calls', async () => {
|
||||
const session = new ExploreSessionState();
|
||||
const target = path.join(testDir, 'internal/usecase/payroll/payslip_builder.go');
|
||||
const original = fs.readFileSync(target, 'utf-8');
|
||||
try {
|
||||
const first = await explore(QUERY, session);
|
||||
expect(fencedLines(first).has('internal/usecase/payroll/payslip_builder.go')).toBe(true);
|
||||
|
||||
fs.writeFileSync(target, original.replace('func sumKind(', 'func sumKindRenamed('), 'utf-8');
|
||||
const second = await explore(QUERY, session);
|
||||
|
||||
// The edited file is served again, whole — a pointer here would send the
|
||||
// agent to a copy of the file that no longer exists.
|
||||
const pointerLines = second.split('\n').filter((l) => l.includes(POINTER));
|
||||
expect(pointerLines.some((l) => l.includes('payslip_builder.go'))).toBe(false);
|
||||
expect(fencedLines(second).get('internal/usecase/payroll/payslip_builder.go')?.size ?? 0)
|
||||
.toBeGreaterThan(20);
|
||||
} finally {
|
||||
fs.writeFileSync(target, original, 'utf-8');
|
||||
}
|
||||
}, 120_000);
|
||||
|
||||
it('always returns real source, even when the session already holds everything', async () => {
|
||||
const session = new ExploreSessionState();
|
||||
await explore(QUERY, session);
|
||||
await explore(QUERY, session);
|
||||
const third = await explore(QUERY, session);
|
||||
const fourth = await explore(QUERY, session);
|
||||
|
||||
// An all-pointer response is the shape that reads as failure. Every call
|
||||
// keeps at least one real fenced block, however much the session holds.
|
||||
for (const [n, text] of [[3, third], [4, fourth]] as const) {
|
||||
const lines = [...fencedLines(text).values()].reduce((s, set) => s + set.size, 0);
|
||||
expect(lines, `call ${n} returned no source at all`).toBeGreaterThan(10);
|
||||
}
|
||||
}, 180_000);
|
||||
|
||||
it('leaves the first call of a session untouched', async () => {
|
||||
const tracked = await explore(QUERY, new ExploreSessionState());
|
||||
const untracked = await explore(QUERY);
|
||||
expect(tracked).toBe(untracked);
|
||||
}, 120_000);
|
||||
|
||||
it('keeps two sessions on one handler independent', async () => {
|
||||
const a = new ExploreSessionState();
|
||||
const b = new ExploreSessionState();
|
||||
const firstForA = await explore(QUERY, a);
|
||||
await explore(QUERY, a);
|
||||
// B's first call has seen nothing, whatever A has been served.
|
||||
expect(await explore(QUERY, b)).toBe(firstForA);
|
||||
}, 180_000);
|
||||
|
||||
it('is off entirely under CODEGRAPH_EXPLORE_DEDUP=0', async () => {
|
||||
const session = new ExploreSessionState();
|
||||
const previous = process.env.CODEGRAPH_EXPLORE_DEDUP;
|
||||
process.env.CODEGRAPH_EXPLORE_DEDUP = '0';
|
||||
try {
|
||||
const first = await explore(QUERY, session);
|
||||
const second = await explore(QUERY, session);
|
||||
expect(second).toBe(first);
|
||||
expect(second).not.toContain(POINTER);
|
||||
} finally {
|
||||
if (previous === undefined) delete process.env.CODEGRAPH_EXPLORE_DEDUP;
|
||||
else process.env.CODEGRAPH_EXPLORE_DEDUP = previous;
|
||||
}
|
||||
}, 120_000);
|
||||
|
||||
it('reports the reclaimed bytes through the CG-4 diagnostic', async () => {
|
||||
const sidecar = path.join(testDir, 'cg18-diagnostic.jsonl');
|
||||
const session = new ExploreSessionState();
|
||||
const previous = process.env.CODEGRAPH_EXPLORE_DEBUG;
|
||||
process.env.CODEGRAPH_EXPLORE_DEBUG = sidecar;
|
||||
try {
|
||||
await explore(QUERY, session);
|
||||
await explore(QUERY, session);
|
||||
} finally {
|
||||
if (previous === undefined) delete process.env.CODEGRAPH_EXPLORE_DEBUG;
|
||||
else process.env.CODEGRAPH_EXPLORE_DEBUG = previous;
|
||||
}
|
||||
const [one, two] = fs.readFileSync(sidecar, 'utf-8').trim().split('\n').map((l) => JSON.parse(l));
|
||||
|
||||
expect(one.dedup.savedChars).toBe(0);
|
||||
expect(two.dedup.savedChars).toBeGreaterThan(1000);
|
||||
expect(two.dedup.backReferenced.length).toBeGreaterThan(0);
|
||||
|
||||
// The reclamation is legible file by file: a back-referenced file spent
|
||||
// none of its reservation, and the response still filled its envelope.
|
||||
const backref = two.files.filter((f: { render: string }) => f.render === 'backref');
|
||||
for (const f of backref) {
|
||||
expect(f.emittedChars).toBe(0);
|
||||
expect(f.dedupSavedChars).toBeGreaterThan(0);
|
||||
expect(f.dedupCovered.length).toBeGreaterThan(0);
|
||||
}
|
||||
const spentOnFreshSource = two.files.reduce((s: number, f: { emittedChars: number }) => s + f.emittedChars, 0);
|
||||
expect(spentOnFreshSource).toBeGreaterThan(0);
|
||||
}, 120_000);
|
||||
});
|
||||
@@ -0,0 +1,207 @@
|
||||
/**
|
||||
* Regression gate for DECLARATION-ONLY files in explore ranking (task CG-28).
|
||||
*
|
||||
* A file that holds nothing but type declarations — an ambient `.d.ts`, vendored
|
||||
* typings, a `types.ts` of pure interfaces — cannot answer a FLOW question: no
|
||||
* bodies, no call edges, no behaviour. But the identifiers it declares are
|
||||
* exactly the generic ones a prose question uses (`Body`, `Message`,
|
||||
* `ImageMetadata`, `ReadableStream`), so on term overlap it out-scored the
|
||||
* implementation and took the envelope. Measured on this fixture before the fix:
|
||||
* rank #1 and 51% of delivered source on a prose flow query.
|
||||
*
|
||||
* CG-25 already covers the file that STARTED this — a Wrangler
|
||||
* `worker-configuration.d.ts`, which announces itself with a generated banner.
|
||||
* `docs/benchmarks/explore-declaration-only-cg28.md` has that measurement; the
|
||||
* banner alone is worth 15–46 points of envelope share. What it does not cover
|
||||
* is a declaration file with no banner at all, which is what this fixture's
|
||||
* `platform-shims.d.ts` is, and what the damping in `rankPenalty` addresses.
|
||||
*
|
||||
* Two claims, and BOTH have to hold — the counter-case is why the penalty is
|
||||
* guarded rather than flat:
|
||||
*
|
||||
* 1. a prose flow query must not let a declaration-only file outrank the
|
||||
* implementation files that answer it;
|
||||
* 2. a query genuinely ABOUT a declared type must still reach the declaration
|
||||
* at full weight.
|
||||
*
|
||||
* The suppression the issue explicitly forbids is also pinned: a damped file is
|
||||
* still a candidate and still named in the response, so one follow-up explore
|
||||
* fetches it.
|
||||
*/
|
||||
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import CodeGraph from '../src/index';
|
||||
import { ToolHandler } from '../src/mcp/tools';
|
||||
import type { ExploreDiagnosticReport, ExploreDiagnosticFile } from '../src/mcp/explore-diagnostics';
|
||||
|
||||
const FIXTURE_SRC = path.join(__dirname, 'fixtures', 'ambient-decls-ts');
|
||||
|
||||
/** Declaration-only, hand-written, NO generated banner — the surviving gap. */
|
||||
const HANDWRITTEN_DECL = 'types/platform-shims.d.ts';
|
||||
/** Declaration-only WITH a Wrangler banner — the CG-25 control in the same run. */
|
||||
const GENERATED_DECL = 'types/worker-configuration.d.ts';
|
||||
/** Declaration-only but IMPORTED by the storage layer — must never be damped. */
|
||||
const SHARED_TYPES = 'src/storage/types.ts';
|
||||
|
||||
/** Prose, naming no symbol — the query shape that let the original file in. */
|
||||
const FLOW_QUERY =
|
||||
'how does an upload request stream the file body to storage and record image metadata';
|
||||
/** Prose that DOES name a declared type — the counter-case. */
|
||||
const TYPE_QUERY = 'what does the UploadStorage interface declare for putting an object';
|
||||
|
||||
describe('CG-28 — a declaration-only file does not outrank implementation on a flow query', () => {
|
||||
let testDir: string;
|
||||
let cg: CodeGraph;
|
||||
let sidecar: string;
|
||||
|
||||
/** One explore call; returns its diagnostic report plus the response text. */
|
||||
const explore = async (query: string): Promise<{ report: ExploreDiagnosticReport; text: string }> => {
|
||||
fs.rmSync(sidecar, { force: true });
|
||||
const previous = process.env.CODEGRAPH_EXPLORE_DEBUG;
|
||||
process.env.CODEGRAPH_EXPLORE_DEBUG = sidecar;
|
||||
let text: string;
|
||||
try {
|
||||
text = (await new ToolHandler(cg).execute('codegraph_explore', { query })).content?.[0]?.text ?? '';
|
||||
} finally {
|
||||
if (previous === undefined) delete process.env.CODEGRAPH_EXPLORE_DEBUG;
|
||||
else process.env.CODEGRAPH_EXPLORE_DEBUG = previous;
|
||||
}
|
||||
const written = fs.readFileSync(sidecar, 'utf-8').trim().split('\n').filter(Boolean);
|
||||
return { report: JSON.parse(written[written.length - 1]!) as ExploreDiagnosticReport, text };
|
||||
};
|
||||
|
||||
const fileOf = (report: ExploreDiagnosticReport, p: string): ExploreDiagnosticFile | undefined =>
|
||||
report.files.find((f) => f.path === p);
|
||||
|
||||
let flow: { report: ExploreDiagnosticReport; text: string };
|
||||
let typed: { report: ExploreDiagnosticReport; text: string };
|
||||
|
||||
beforeAll(async () => {
|
||||
testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cg28-'));
|
||||
fs.cpSync(FIXTURE_SRC, testDir, { recursive: true });
|
||||
fs.rmSync(path.join(testDir, '.codegraph'), { recursive: true, force: true });
|
||||
sidecar = path.join(testDir, 'explore-diag.jsonl');
|
||||
|
||||
cg = CodeGraph.initSync(testDir);
|
||||
await cg.indexAll();
|
||||
|
||||
flow = await explore(FLOW_QUERY);
|
||||
typed = await explore(TYPE_QUERY);
|
||||
}, 120_000);
|
||||
|
||||
afterAll(() => {
|
||||
if (cg) cg.destroy();
|
||||
if (testDir && fs.existsSync(testDir)) fs.rmSync(testDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
describe('fixture shape — if this rots, the gate below means nothing', () => {
|
||||
it('holds two declaration-only files that differ only in the banner', () => {
|
||||
for (const p of [HANDWRITTEN_DECL, GENERATED_DECL]) {
|
||||
const nodes = cg.getNodesInFile(p).filter((n) => n.kind !== 'file' && n.kind !== 'import');
|
||||
expect(nodes.length, `${p} declares nothing`).toBeGreaterThan(10);
|
||||
// Every symbol type-level, nothing with a body — the structural test the
|
||||
// penalty keys on. A `function`/`class` creeping in would silently exempt
|
||||
// the file and make every assertion below vacuous.
|
||||
expect(nodes.every((n) => n.kind === 'interface' || n.kind === 'type_alias'), `${p} has a non-type symbol`).toBe(true);
|
||||
}
|
||||
// Only one of them announces itself, so the CG-25 penalty is the ONLY
|
||||
// difference between the two — that is what makes them comparable.
|
||||
expect(cg.getFile(GENERATED_DECL)?.generated).toBe(true);
|
||||
expect(cg.getFile(HANDWRITTEN_DECL)?.generated).toBeFalsy();
|
||||
});
|
||||
|
||||
it('holds a pure-type module the code IMPORTS, as the safety control', () => {
|
||||
// Identical to the ambient files on kinds and bodies; different only in
|
||||
// that the storage layer is typed by it. This is the shape the penalty
|
||||
// must NOT catch — a `types.ts` the codebase depends on is part of the
|
||||
// structure of any answer about that code.
|
||||
const nodes = cg.getNodesInFile(SHARED_TYPES).filter((n) => n.kind !== 'file' && n.kind !== 'import');
|
||||
expect(nodes.length).toBeGreaterThan(0);
|
||||
expect(nodes.every((n) => n.kind === 'interface' || n.kind === 'type_alias')).toBe(true);
|
||||
expect(cg.getFile(SHARED_TYPES)?.generated).toBeFalsy();
|
||||
});
|
||||
|
||||
it('holds implementation files that DO answer the flow question', () => {
|
||||
for (const p of ['src/routes/upload.ts', 'src/storage/stream.ts', 'src/storage/metadata.ts']) {
|
||||
expect(cg.getNodesInFile(p).some((n) => n.kind === 'function'), `${p} has no functions`).toBe(true);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('the gate — a prose flow query', () => {
|
||||
it('damps the un-bannered declaration file rather than letting it rank free', () => {
|
||||
const rec = fileOf(flow.report, HANDWRITTEN_DECL);
|
||||
expect(rec, 'the declaration file is not even a candidate — fixture drifted').toBeDefined();
|
||||
expect(rec!.ambientDeclaration).toBe(true);
|
||||
expect(rec!.penalty).toBeLessThan(1);
|
||||
});
|
||||
|
||||
it('does not let it outrank the implementation files', () => {
|
||||
const decl = fileOf(flow.report, HANDWRITTEN_DECL)!;
|
||||
const impl = flow.report.files.filter((f) => f.path.startsWith('src/') && f.finalChars > 0);
|
||||
expect(impl.length, 'no implementation file delivered anything').toBeGreaterThanOrEqual(2);
|
||||
// Measured before the fix: the declaration file was rank #1 with score 53
|
||||
// against the best implementation file's 34. The bar is that at least one
|
||||
// implementation file now ranks above it — ordinary budget movement must
|
||||
// not fail the suite, but the inversion coming back must.
|
||||
expect(impl.some((f) => f.rank < decl.rank), 'declaration file still ranks first').toBe(true);
|
||||
});
|
||||
|
||||
it('still names it in the response, so one follow-up call fetches it', () => {
|
||||
// The issue forbids suppression: a damped file must remain reachable.
|
||||
expect(flow.text).toContain(HANDWRITTEN_DECL);
|
||||
});
|
||||
|
||||
it('leaves the implementation files at full weight', () => {
|
||||
for (const f of flow.report.files.filter((x) => x.path.startsWith('src/'))) {
|
||||
expect(f.ambientDeclaration, `${f.path} was misread as an ambient declaration`).toBe(false);
|
||||
expect(f.penalty).toBe(1);
|
||||
}
|
||||
});
|
||||
|
||||
it('does not damp a pure-type module the codebase imports', () => {
|
||||
// The condition that keeps this narrow enough to be safe. Without it the
|
||||
// same rule demotes `displacement-ts`'s pipeline `types.ts` — pure
|
||||
// interfaces, but 13 inbound imports — and breaks the CG-31 gate.
|
||||
const rec = flow.report.files.find((f) => f.path === SHARED_TYPES);
|
||||
if (rec) {
|
||||
expect(rec.ambientDeclaration, `${SHARED_TYPES} was flagged ambient`).toBe(false);
|
||||
expect(rec.penalty).toBe(1);
|
||||
}
|
||||
// Independent of whether this query ranked it: the predicate itself must
|
||||
// separate the two shapes.
|
||||
const isAmbient = cg.ambientDeclarationFilePredicate([SHARED_TYPES, HANDWRITTEN_DECL]);
|
||||
expect(isAmbient(SHARED_TYPES)).toBe(false);
|
||||
expect(isAmbient(HANDWRITTEN_DECL)).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('the counter-case — a query that NAMES a declared type', () => {
|
||||
it('reaches the declaration at full weight, undamped', () => {
|
||||
const rec = fileOf(typed.report, HANDWRITTEN_DECL);
|
||||
expect(rec, 'the named type\'s file is not a candidate').toBeDefined();
|
||||
expect(rec!.ambientDeclaration).toBe(true);
|
||||
// Detected as declaration-only, but EXEMPT — the query asked for it.
|
||||
expect(rec!.penalty).toBe(1);
|
||||
});
|
||||
|
||||
it('ranks it first and delivers its source', () => {
|
||||
const rec = fileOf(typed.report, HANDWRITTEN_DECL)!;
|
||||
expect(rec.rank).toBe(1);
|
||||
expect(rec.finalChars).toBeGreaterThan(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('the two penalties do not stack', () => {
|
||||
it('charges a generated declaration file once, at the stronger rate', () => {
|
||||
// A file that is BOTH generated and declaration-only has ONE property two
|
||||
// signals happen to see. Penalising twice (0.3 * 0.5 = 0.15) is how a file
|
||||
// gets cliffed out of answers where it is genuinely relevant.
|
||||
const rec = flow.report.files.find((f) => f.generated && f.ambientDeclaration);
|
||||
if (!rec) return; // not a candidate for this query — nothing to assert
|
||||
expect(rec.penalty).toBeGreaterThanOrEqual(0.3);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,303 @@
|
||||
/**
|
||||
* Per-file allocation diagnostic for codegraph_explore (CG-4).
|
||||
*
|
||||
* The instrument ships in the product binary, so the load-bearing property is
|
||||
* NOT what it reports — it's that it reports NOTHING unless asked. An explore
|
||||
* response is the agent's context; a diagnostic that perturbs it by one byte
|
||||
* invalidates every A/B measurement taken with it on, which is the exact thing
|
||||
* the rest of the budget-allocation work depends on.
|
||||
*
|
||||
* So the first block pins byte-identical output across on/off, and only then
|
||||
* do we assert the report's shape and internal consistency.
|
||||
*/
|
||||
import { describe, it, expect, beforeAll, afterAll, afterEach, vi } from 'vitest';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import { ToolHandler } from '../src/mcp/tools';
|
||||
import { attributeSourceBytes } from '../src/mcp/explore-diagnostics';
|
||||
import CodeGraph from '../src/index';
|
||||
|
||||
const DEBUG_ENV = 'CODEGRAPH_EXPLORE_DEBUG';
|
||||
|
||||
/** Restore the env var to "unset" — `delete` matters; '' is a distinct case. */
|
||||
function clearDebugEnv(): void {
|
||||
delete process.env[DEBUG_ENV];
|
||||
}
|
||||
|
||||
describe('attributeSourceBytes', () => {
|
||||
it('attributes a fenced block to the file section header above it', () => {
|
||||
const text = [
|
||||
'**Exploration: x**',
|
||||
'',
|
||||
'**`src/a.ts`** — foo(function)',
|
||||
'',
|
||||
'```typescript',
|
||||
'1\tconst a = 1;',
|
||||
'2\tconst b = 2;',
|
||||
'```',
|
||||
'',
|
||||
'**`src/b.ts`** — bar(function)',
|
||||
'',
|
||||
'```typescript',
|
||||
'1\tconst c = 3;',
|
||||
'```',
|
||||
'',
|
||||
].join('\n');
|
||||
const bytes = attributeSourceBytes(text);
|
||||
expect(bytes.get('src/a.ts')).toBe('1\tconst a = 1;\n2\tconst b = 2;'.length);
|
||||
expect(bytes.get('src/b.ts')).toBe('1\tconst c = 3;'.length);
|
||||
});
|
||||
|
||||
it('sums multiple fenced blocks under one file header', () => {
|
||||
const text = [
|
||||
'**`src/a.ts`** — foo(function)',
|
||||
'',
|
||||
'```ts',
|
||||
'aa',
|
||||
'```',
|
||||
'',
|
||||
'```ts',
|
||||
'bbb',
|
||||
'```',
|
||||
].join('\n');
|
||||
expect(attributeSourceBytes(text).get('src/a.ts')).toBe('aa'.length + 'bbb'.length);
|
||||
});
|
||||
|
||||
it('counts an unterminated block — the ceiling can cut mid-fence', () => {
|
||||
const text = ['**`src/a.ts`** — foo(function)', '', '```ts', 'x'.repeat(40)].join('\n');
|
||||
expect(attributeSourceBytes(text).get('src/a.ts')).toBe(40);
|
||||
});
|
||||
|
||||
it('returns nothing for text with no file sections', () => {
|
||||
expect(attributeSourceBytes('No relevant code found for "zzz"').size).toBe(0);
|
||||
expect(attributeSourceBytes('').size).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('codegraph_explore allocation diagnostic', () => {
|
||||
let testDir: string;
|
||||
let sidecarDir: string;
|
||||
let cg: CodeGraph;
|
||||
let handler: ToolHandler;
|
||||
|
||||
const QUERY = 'Session method helper callSession';
|
||||
|
||||
beforeAll(async () => {
|
||||
testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-explore-diag-'));
|
||||
sidecarDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-explore-diag-out-'));
|
||||
const srcDir = path.join(testDir, 'src');
|
||||
fs.mkdirSync(srcDir);
|
||||
|
||||
// One fat file plus several small callers, so the render loop exercises
|
||||
// more than one allocation branch (clusters for the fat file, whole-file
|
||||
// for the small ones) and there is a real per-file split to report.
|
||||
const fatLines: string[] = ['export class Session {'];
|
||||
for (let i = 0; i < 30; i++) {
|
||||
fatLines.push(` method${i}(arg: string): string {`);
|
||||
fatLines.push(` return this.helper${i}(arg) + "${i}";`);
|
||||
fatLines.push(` }`);
|
||||
fatLines.push(` private helper${i}(arg: string): string {`);
|
||||
fatLines.push(` return arg.repeat(${i + 1});`);
|
||||
fatLines.push(` }`);
|
||||
}
|
||||
fatLines.push('}');
|
||||
fs.writeFileSync(path.join(srcDir, 'session.ts'), fatLines.join('\n'));
|
||||
|
||||
for (let i = 0; i < 6; i++) {
|
||||
fs.writeFileSync(
|
||||
path.join(srcDir, `support${i}.ts`),
|
||||
`import { Session } from './session';\n` +
|
||||
`export function callSession${i}(s: Session) {\n` +
|
||||
` return s.method${i}('hi');\n` +
|
||||
`}\n`,
|
||||
);
|
||||
}
|
||||
|
||||
clearDebugEnv();
|
||||
cg = CodeGraph.initSync(testDir, { config: { include: ['**/*.ts'], exclude: [] } });
|
||||
await cg.indexAll();
|
||||
handler = new ToolHandler(cg);
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
clearDebugEnv();
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
clearDebugEnv();
|
||||
if (cg) cg.destroy();
|
||||
for (const dir of [testDir, sidecarDir]) {
|
||||
if (dir && fs.existsSync(dir)) fs.rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
const explore = async (): Promise<string> => {
|
||||
const result = await handler.execute('codegraph_explore', { query: QUERY });
|
||||
return result.content?.[0]?.text ?? '';
|
||||
};
|
||||
|
||||
it('produces byte-identical output whether the diagnostic is on or off', async () => {
|
||||
clearDebugEnv();
|
||||
const off = await explore();
|
||||
expect(off.length).toBeGreaterThan(0);
|
||||
|
||||
// Sanity: the tool itself is deterministic, so a difference below is
|
||||
// attributable to the diagnostic and not to explore's own variance.
|
||||
expect(await explore()).toBe(off);
|
||||
|
||||
vi.spyOn(process.stderr, 'write').mockImplementation((() => true) as typeof process.stderr.write);
|
||||
const sidecar = path.join(sidecarDir, 'identical.jsonl');
|
||||
for (const value of ['1', 'json', sidecar]) {
|
||||
process.env[DEBUG_ENV] = value;
|
||||
const on = await explore();
|
||||
clearDebugEnv();
|
||||
expect(on).toBe(off);
|
||||
}
|
||||
});
|
||||
|
||||
it('writes nothing to stderr when the env var is unset', async () => {
|
||||
clearDebugEnv();
|
||||
const writes: string[] = [];
|
||||
vi.spyOn(process.stderr, 'write').mockImplementation(((chunk: unknown) => {
|
||||
writes.push(String(chunk));
|
||||
return true;
|
||||
}) as typeof process.stderr.write);
|
||||
await explore();
|
||||
expect(writes.join('')).toBe('');
|
||||
});
|
||||
|
||||
it('stays off for every falsy env value', async () => {
|
||||
const writes: string[] = [];
|
||||
vi.spyOn(process.stderr, 'write').mockImplementation(((chunk: unknown) => {
|
||||
writes.push(String(chunk));
|
||||
return true;
|
||||
}) as typeof process.stderr.write);
|
||||
for (const value of ['', '0', 'false', 'off', 'no', 'OFF', ' 0 ']) {
|
||||
process.env[DEBUG_ENV] = value;
|
||||
await explore();
|
||||
}
|
||||
expect(writes.join('')).toBe('');
|
||||
});
|
||||
|
||||
it('prints a per-file table to stderr when enabled', async () => {
|
||||
const writes: string[] = [];
|
||||
vi.spyOn(process.stderr, 'write').mockImplementation(((chunk: unknown) => {
|
||||
writes.push(String(chunk));
|
||||
return true;
|
||||
}) as typeof process.stderr.write);
|
||||
|
||||
process.env[DEBUG_ENV] = '1';
|
||||
await explore();
|
||||
const out = writes.join('');
|
||||
|
||||
expect(out).toContain('codegraph explore diagnostic');
|
||||
// Totals: envelope vs budget, and the file-selection funnel with its floor.
|
||||
expect(out).toMatch(/envelope [\d,]+ chars delivered · [\d,]+ allocated of [\d,]+ budget/);
|
||||
expect(out).toMatch(/hard ceiling [\d,]+/);
|
||||
// The funnel runs low-value filter → score floor; the floor is fractional
|
||||
// now that scoring is kind-weighted (CG-10).
|
||||
expect(out).toMatch(
|
||||
/files [\d,]+ grouped .*past low-value filter .*past score floor \(>=[\d.]+\).*in output \(maxFiles \d+\)/,
|
||||
);
|
||||
// Per-file columns.
|
||||
expect(out).toMatch(/#\s+alloc%\s+deliv%\s+bytes\s+reserved\s+score\s+graph\s+hits\s+pen\s+flags\s+render\s+file/);
|
||||
// The proportional split (CG-12): what was reserved, and where the cliff fell.
|
||||
expect(out).toMatch(/allocation [\d,]+ reserved of [\d,]+ pool · cliff at weight [\d.]+/);
|
||||
expect(out).toContain('src/session.ts');
|
||||
expect(out).toMatch(/\d+\.\d%/);
|
||||
// Kind mix — what each file's score was bought with.
|
||||
expect(out).toMatch(/kinds: (?:\w+:\d+ ?)+/);
|
||||
});
|
||||
|
||||
it('appends one JSON report per call to a sidecar path', async () => {
|
||||
const sidecar = path.join(sidecarDir, 'reports.jsonl');
|
||||
process.env[DEBUG_ENV] = sidecar;
|
||||
await explore();
|
||||
await explore();
|
||||
clearDebugEnv();
|
||||
|
||||
const rows = fs.readFileSync(sidecar, 'utf-8').trim().split('\n');
|
||||
expect(rows).toHaveLength(2);
|
||||
|
||||
const report = JSON.parse(rows[0]!);
|
||||
expect(report.tool).toBe('codegraph_explore');
|
||||
expect(report.query).toBe(QUERY);
|
||||
|
||||
// Totals the task asks for: envelope vs maxOutputChars, files considered
|
||||
// vs included, and the score floor that was applied.
|
||||
expect(report.budget.maxOutputChars).toBeGreaterThan(0);
|
||||
expect(report.envelope.chars).toBeGreaterThan(0);
|
||||
expect(report.selection.scoreFloor).toBeGreaterThan(0);
|
||||
expect(report.selection.filesGrouped).toBeGreaterThanOrEqual(report.selection.filesPastLowValueFilter);
|
||||
expect(report.selection.filesPastLowValueFilter).toBeGreaterThanOrEqual(report.selection.filesPastScoreFloor);
|
||||
expect(report.selection.filesPastScoreFloor).toBeGreaterThanOrEqual(report.selection.filesRanked);
|
||||
expect(report.selection.filesRanked).toBeGreaterThanOrEqual(report.selection.filesInFinalOutput);
|
||||
expect(report.selection.filesInFinalOutput).toBeGreaterThan(0);
|
||||
expect(report.selection.filesInFinalOutput).toBeLessThanOrEqual(report.budget.maxFiles);
|
||||
|
||||
// Per-file: score, bytes, share, clipped, spine.
|
||||
const shown = report.files.filter((f: { finalChars: number }) => f.finalChars > 0);
|
||||
expect(shown.length).toBeGreaterThan(0);
|
||||
for (const f of shown) {
|
||||
expect(typeof f.path).toBe('string');
|
||||
expect(typeof f.score).toBe('number');
|
||||
expect(typeof f.graphScore).toBe('number');
|
||||
expect(typeof f.clipped).toBe('boolean');
|
||||
expect(typeof f.spine).toBe('boolean');
|
||||
expect(f.finalChars).toBeGreaterThan(0);
|
||||
expect(f.share).toBeGreaterThan(0);
|
||||
expect(f.share).toBeLessThanOrEqual(1);
|
||||
expect(f.render).toBeTruthy();
|
||||
}
|
||||
expect(shown.some((f: { path: string }) => f.path === 'src/session.ts')).toBe(true);
|
||||
});
|
||||
|
||||
it('attributes the envelope consistently — per-file bytes sum to the reported source total', async () => {
|
||||
const sidecar = path.join(sidecarDir, 'consistency.jsonl');
|
||||
process.env[DEBUG_ENV] = sidecar;
|
||||
const text = await explore();
|
||||
clearDebugEnv();
|
||||
|
||||
const report = JSON.parse(fs.readFileSync(sidecar, 'utf-8').trim());
|
||||
expect(report.envelope.chars).toBe(text.length);
|
||||
|
||||
const summed = report.files.reduce(
|
||||
(s: number, f: { finalChars: number }) => s + f.finalChars, 0,
|
||||
);
|
||||
expect(summed).toBe(report.envelope.sourceChars);
|
||||
expect(report.envelope.sourceChars + report.envelope.metaChars).toBe(report.envelope.chars);
|
||||
// Shares are fractions of the delivered envelope, so they can't exceed it.
|
||||
const shareSum = report.files.reduce((s: number, f: { share: number }) => s + f.share, 0);
|
||||
expect(shareSum).toBeLessThanOrEqual(1.0001);
|
||||
expect(shareSum).toBeCloseTo(report.envelope.sourceShare, 3);
|
||||
});
|
||||
|
||||
it('survives an unwritable sink without failing the explore call', async () => {
|
||||
clearDebugEnv();
|
||||
const expected = await explore();
|
||||
|
||||
// A directory is never a valid append target.
|
||||
process.env[DEBUG_ENV] = sidecarDir;
|
||||
const result = await handler.execute('codegraph_explore', { query: QUERY });
|
||||
clearDebugEnv();
|
||||
|
||||
expect(result.isError).toBeFalsy();
|
||||
expect(result.content?.[0]?.text).toBe(expected);
|
||||
});
|
||||
|
||||
it('records a report even when explore finds nothing', async () => {
|
||||
const sidecar = path.join(sidecarDir, 'empty.jsonl');
|
||||
process.env[DEBUG_ENV] = sidecar;
|
||||
const result = await handler.execute('codegraph_explore', {
|
||||
query: 'zzzznonexistentsymbolzzzz',
|
||||
});
|
||||
clearDebugEnv();
|
||||
|
||||
expect(result.content?.[0]?.text).toContain('No relevant code found');
|
||||
const report = JSON.parse(fs.readFileSync(sidecar, 'utf-8').trim());
|
||||
expect(report.note).toContain('no relevant code found');
|
||||
expect(report.files).toEqual([]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,245 @@
|
||||
/**
|
||||
* Regression fixture for CG-31 — a clustered render may not spend a reservation
|
||||
* still owed to a file the loop has not reached.
|
||||
*
|
||||
* The allocator hands every admitted file a reservation (CG-12), and the render
|
||||
* loop then walks the files in rank order. Carry-forward slack lets a file spend
|
||||
* what the files ABOVE it left on the table, which is right; what was missing is
|
||||
* the other half — nothing was held back for the files BELOW it. The whole-file
|
||||
* BUY arm has always refused that trade (`owedBelow`, `tools.ts`); the cluster
|
||||
* path had no equivalent, so `fileBudget`/`SPINE_CEILING` read what was left
|
||||
* before the hard ceiling rather than what was still promised, and the first
|
||||
* oversize file could take the response.
|
||||
*
|
||||
* `__tests__/fixtures/displacement-ts/` reproduces it. Four pipeline stages
|
||||
* compete for one envelope; the first, `ingest.ts`, is a single ~20K function —
|
||||
* one cluster member far bigger than any reservation it can earn — so it takes
|
||||
* the bounded overshoot CG-30 left it. The fixture is padded to >500 indexed
|
||||
* files on purpose: the displacement only exists on the 24K tier, where the
|
||||
* reservations plus the response preamble genuinely saturate the hard ceiling.
|
||||
*
|
||||
* Measured against the pre-fix build (CG-30 landed, CG-31 not):
|
||||
*
|
||||
* ingest.ts 9,301 chars emitted on a 6,289 spendable — then dropped whole
|
||||
* by the final ceiling, so it cost the response and delivered 0
|
||||
* types.ts skipped `budget-whole-file`
|
||||
* sink.ts skipped `budget-whole-file`
|
||||
* delivered 3 of 6 admitted files, 14,908-char envelope
|
||||
*
|
||||
* With the guard: 6 of 6, 22,066-char envelope, and `ingest.ts` bounded to the
|
||||
* 4,913 that were actually still free.
|
||||
*/
|
||||
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import CodeGraph from '../src/index';
|
||||
import { ToolHandler } from '../src/mcp/tools';
|
||||
import { attributeSourceBytes } from '../src/mcp/explore-diagnostics';
|
||||
import type { ExploreDiagnosticReport, ExploreDiagnosticFile } from '../src/mcp/explore-diagnostics';
|
||||
|
||||
const FIXTURE_SRC = path.join(__dirname, 'fixtures', 'displacement-ts');
|
||||
|
||||
/**
|
||||
* Padding modules, written into the temp copy rather than checked in. The
|
||||
* output tier is chosen by INDEXED FILE COUNT, and the displacement this test
|
||||
* pins only exists at >=500 files (24K envelope against a 24.4K render ceiling
|
||||
* that also has to hold the response preamble). Below that the ceiling has
|
||||
* enough slack to absorb an overshoot and the bug is invisible.
|
||||
*/
|
||||
const FILLER_FILES = 520;
|
||||
|
||||
/** A symbol bag spanning all four stages — they compete for one envelope. */
|
||||
const QUERY = 'ingestRecords normalizeRecords enrichRecords publishRecords';
|
||||
/** One symbol, one file — the concentration case the guard must not flatten. */
|
||||
const PRECISE_QUERY = 'ingestRecords';
|
||||
|
||||
/** The giant: one ~20K function, the file that used to take the response. */
|
||||
const GIANT = 'src/pipeline/ingest.ts';
|
||||
/** Ranked below the giant and dropped by it pre-fix. */
|
||||
const STARVED = ['src/pipeline/types.ts', 'src/pipeline/sink.ts'];
|
||||
|
||||
interface Probe {
|
||||
response: string;
|
||||
report: ExploreDiagnosticReport;
|
||||
bytes: Map<string, number>;
|
||||
}
|
||||
|
||||
describe('CG-31 — the cluster path holds back what is still owed below it', () => {
|
||||
let testDir: string;
|
||||
let cg: CodeGraph;
|
||||
let spread: Probe;
|
||||
let precise: Probe;
|
||||
|
||||
const fileOf = (probe: Probe, p: string): ExploreDiagnosticFile => {
|
||||
const rec = probe.report.files.find((f) => f.path === p);
|
||||
if (!rec) throw new Error(`${p} absent from the diagnostic report`);
|
||||
return rec;
|
||||
};
|
||||
/** Admitted = the allocator reserved bytes for it. */
|
||||
const admitted = (probe: Probe): ExploreDiagnosticFile[] =>
|
||||
probe.report.files.filter((f) => (f.allowance ?? 0) > 0);
|
||||
|
||||
beforeAll(async () => {
|
||||
testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cg31-'));
|
||||
fs.cpSync(FIXTURE_SRC, testDir, { recursive: true });
|
||||
fs.rmSync(path.join(testDir, '.codegraph'), { recursive: true, force: true });
|
||||
|
||||
const filler = path.join(testDir, 'src', 'generated');
|
||||
fs.mkdirSync(filler, { recursive: true });
|
||||
for (let i = 0; i < FILLER_FILES; i++) {
|
||||
// Deterministic, unrelated to the query — these pad the file count, they
|
||||
// must never rank.
|
||||
fs.writeFileSync(
|
||||
path.join(filler, `unit${i}.ts`),
|
||||
`export const seed${i} = ${i};\n`
|
||||
+ `export function widget${i}(n: number): number {\n return n * ${i + 1} + seed${i};\n}\n`,
|
||||
);
|
||||
}
|
||||
|
||||
cg = CodeGraph.initSync(testDir);
|
||||
await cg.indexAll();
|
||||
|
||||
// The per-file bounds are only observable through the diagnostic sidecar.
|
||||
const sidecar = path.join(testDir, 'explore-diag.jsonl');
|
||||
const previous = process.env.CODEGRAPH_EXPLORE_DEBUG;
|
||||
process.env.CODEGRAPH_EXPLORE_DEBUG = sidecar;
|
||||
const run = async (handler: ToolHandler, query: string): Promise<Probe> => {
|
||||
const result = await handler.execute('codegraph_explore', { query });
|
||||
const response = result.content?.[0]?.text ?? '';
|
||||
const written = fs.readFileSync(sidecar, 'utf-8').trim().split('\n').filter(Boolean);
|
||||
return {
|
||||
response,
|
||||
report: JSON.parse(written[written.length - 1]!) as ExploreDiagnosticReport,
|
||||
bytes: attributeSourceBytes(response),
|
||||
};
|
||||
};
|
||||
try {
|
||||
const handler = new ToolHandler(cg);
|
||||
spread = await run(handler, QUERY);
|
||||
precise = await run(handler, PRECISE_QUERY);
|
||||
} finally {
|
||||
if (previous === undefined) delete process.env.CODEGRAPH_EXPLORE_DEBUG;
|
||||
else process.env.CODEGRAPH_EXPLORE_DEBUG = previous;
|
||||
}
|
||||
}, 180_000);
|
||||
|
||||
afterAll(() => {
|
||||
if (cg) cg.destroy();
|
||||
if (testDir && fs.existsSync(testDir)) fs.rmSync(testDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
// ── Fixture shape — if these rot, the gate below means nothing ─────────────
|
||||
|
||||
describe('fixture shape', () => {
|
||||
it('sits on the 24K tier, where the reservations saturate the ceiling', () => {
|
||||
expect(cg.getStats().fileCount).toBeGreaterThanOrEqual(500);
|
||||
expect(spread.report.budget.maxOutputChars).toBe(24000);
|
||||
});
|
||||
|
||||
it('admits every stage file, so there is something to displace', () => {
|
||||
const paths = admitted(spread).map((f) => f.path);
|
||||
expect(paths).toContain(GIANT);
|
||||
for (const p of STARVED) expect(paths).toContain(p);
|
||||
expect(paths.length).toBeGreaterThanOrEqual(5);
|
||||
});
|
||||
|
||||
it('renders the giant through the CLUSTER path, over its reservation', () => {
|
||||
const rec = fileOf(spread, GIANT);
|
||||
expect(rec.render).toBe('clusters');
|
||||
// One member bigger than anything it can earn beside its siblings — the
|
||||
// shape that makes the bounded overshoot fire at all.
|
||||
const source = fs.readFileSync(path.join(testDir, GIANT), 'utf-8');
|
||||
expect(source.length).toBeGreaterThan((rec.spendable ?? 0) * 2);
|
||||
// And the guard actually bit — a vacuous pass here would hide a
|
||||
// regression. Measured against the bounded overshoot a cluster's top
|
||||
// member may otherwise take (1.5x, CG-30), which is what it refused.
|
||||
expect(rec.funded).not.toBeNull();
|
||||
expect(rec.funded!).toBeLessThan(Math.round(rec.spendable! * 1.5));
|
||||
});
|
||||
});
|
||||
|
||||
// ── The gate ──────────────────────────────────────────────────────────────
|
||||
|
||||
describe('displacement refusal', () => {
|
||||
it('CG-31 GATE: no clustered file emits past what was still free to spend', () => {
|
||||
for (const probe of [spread, precise]) {
|
||||
const over = probe.report.files
|
||||
.filter((f) => f.render === 'clusters' && f.funded !== null)
|
||||
// +1 for the render loop's own rounding on the windowed cut.
|
||||
.filter((f) => f.emittedChars > f.funded! + 1)
|
||||
.map((f) => `${f.path}: ${f.emittedChars} of ${f.funded}`);
|
||||
expect(over).toEqual([]);
|
||||
}
|
||||
});
|
||||
|
||||
it('CG-31 GATE: every admitted file below the top one is delivered', () => {
|
||||
// Pre-fix: 3 of 6 — `ingest.ts` overshot, was itself cut by the final
|
||||
// ceiling, and took `types.ts` + `sink.ts` down with it.
|
||||
for (const rec of admitted(spread)) {
|
||||
expect(rec.skipped, `${rec.path} skipped`).toBeNull();
|
||||
expect(spread.bytes.get(rec.path) ?? 0, `${rec.path} bytes`).toBeGreaterThan(0);
|
||||
}
|
||||
for (const p of STARVED) expect(spread.bytes.get(p) ?? 0).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('the guard is symmetric — it is about ORDER, not rank', () => {
|
||||
// Nothing here protects rank #1 specifically: the LAST admitted file, the
|
||||
// only one with no reservation owed below it, is delivered too.
|
||||
const files = admitted(spread);
|
||||
const last = files[files.length - 1]!;
|
||||
expect(last.skipped).toBeNull();
|
||||
expect(spread.bytes.get(last.path) ?? 0).toBeGreaterThan(0);
|
||||
// And the last file is never itself cut by the guard — nothing is owed
|
||||
// below it, so `funded` may not sit under its own reservation.
|
||||
expect(last.funded!).toBeGreaterThanOrEqual(Math.min(last.allowance!, last.emittedChars));
|
||||
});
|
||||
|
||||
it('a kept promise is not a displacement — no file is cut below its reservation', () => {
|
||||
for (const probe of [spread, precise]) {
|
||||
for (const rec of admitted(probe)) {
|
||||
if (rec.funded === null) continue;
|
||||
expect(rec.funded, rec.path).toBeGreaterThanOrEqual(
|
||||
Math.min(rec.allowance!, rec.emittedChars));
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it('nothing is lost to the hard ceiling — the epilogue is cut before a section', () => {
|
||||
// A section thrown away by the final truncation is the same starvation
|
||||
// arriving after the guard has done its work: the bytes were held back
|
||||
// for that file and then nobody received them.
|
||||
for (const probe of [spread, precise]) {
|
||||
expect(probe.report.files.filter((f) => f.render === 'dropped')).toEqual([]);
|
||||
}
|
||||
});
|
||||
|
||||
it('keeps the response inside the hard ceiling', () => {
|
||||
for (const probe of [spread, precise]) {
|
||||
expect(probe.report.envelope.chars).toBeLessThanOrEqual(probe.report.budget.hardCeiling);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ── The thing the guard must NOT become ───────────────────────────────────
|
||||
|
||||
describe('concentration survives', () => {
|
||||
it('a precise symbol query still puts the most source in the named file', () => {
|
||||
const mine = precise.bytes.get(GIANT) ?? 0;
|
||||
const others = [...precise.bytes.entries()].filter(([p]) => p !== GIANT);
|
||||
expect(mine).toBeGreaterThan(0);
|
||||
for (const [p, n] of others) {
|
||||
expect(mine, `${GIANT} vs ${p}`).toBeGreaterThan(n);
|
||||
}
|
||||
// Not a forced even split: the named file takes a clear plurality.
|
||||
const total = [...precise.bytes.values()].reduce((s, n) => s + n, 0);
|
||||
expect(mine / total).toBeGreaterThan(1 / precise.bytes.size);
|
||||
});
|
||||
|
||||
it('the named file still outspends what it would get from an even split', () => {
|
||||
const rec = fileOf(precise, GIANT);
|
||||
const even = precise.report.budget.maxOutputChars / admitted(precise).length;
|
||||
expect(rec.emittedChars).toBeGreaterThan(even);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,157 @@
|
||||
/**
|
||||
* Regression gate for the FACTORY-CLOSURE file shape (task CG-27).
|
||||
*
|
||||
* A `createFoo()` that returns an object of closures spans almost all of its
|
||||
* file, so its indexed range is an ENVELOPE around every symbol the query
|
||||
* actually wants. Svelte 5 rune stores, React custom-hook modules, IIFE
|
||||
* module-pattern JS and Zustand's `create((set, get) => ({ … }))` are all
|
||||
* written this way, so it is a shape rather than a one-repo quirk.
|
||||
*
|
||||
* CG-27 asked whether the >50%-of-file envelope drop — which fires for `class`,
|
||||
* `struct`, `interface` and friends but not for `function`/`method` — should be
|
||||
* extended to cover it. **Measured, it should not**, and the issue was closed as
|
||||
* obsolete: `docs/benchmarks/explore-factory-closure-cg27.md` has the numbers.
|
||||
* Two independent mechanisms already absorb the shape:
|
||||
*
|
||||
* - `shrinkCluster` orders members by (importance desc, SIZE ASC) and refuses
|
||||
* any member that overruns the cap once something is kept, so a file-spanning
|
||||
* member is only ever selected when it is the sole member of the top
|
||||
* importance tier;
|
||||
* - when it IS selected, CG-30 windows it on whole lines rather than emitting
|
||||
* it whole, so the file still delivers bounded, readable source.
|
||||
*
|
||||
* Dropping the range instead SPLITS the file into several clusters, and only the
|
||||
* first-chosen cluster may be shrunk — measured, a trivial 7-line cluster won the
|
||||
* density tiebreak and the answer-bearing cluster was dropped whole, taking the
|
||||
* rank-#1 file from 7,539 chars and 7 of 11 inner definitions to 397 and none.
|
||||
*
|
||||
* So this file pins the OUTCOME, not the mechanism: whatever future work does to
|
||||
* clustering, a factory-closure file must keep delivering the closures inside it
|
||||
* — that is what stops the agent Reading the file back.
|
||||
*/
|
||||
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import CodeGraph from '../src/index';
|
||||
import { ToolHandler } from '../src/mcp/tools';
|
||||
import type { ExploreDiagnosticReport } from '../src/mcp/explore-diagnostics';
|
||||
|
||||
const FIXTURE_SRC = path.join(__dirname, 'fixtures', 'factory-closure-ts');
|
||||
|
||||
/** The factory file, and the closure factory whose body is nearly all of it. */
|
||||
const TARGET = 'src/stores/dashboard-store.ts';
|
||||
const FACTORY = 'createDashboardStore';
|
||||
/** Prose the way a newcomer asks it, naming two of the closures inside. */
|
||||
const QUERY = 'how does the dashboard store refresh its metrics and apply a filter';
|
||||
|
||||
describe('CG-27 — a factory-closure file delivers the closures inside it', () => {
|
||||
let testDir: string;
|
||||
let cg: CodeGraph;
|
||||
let response: string;
|
||||
let report: ExploreDiagnosticReport;
|
||||
/** Source lines of TARGET the response actually carried. */
|
||||
let delivered: Set<number>;
|
||||
let sourceLines: string[];
|
||||
|
||||
beforeAll(async () => {
|
||||
testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cg27-'));
|
||||
fs.cpSync(FIXTURE_SRC, testDir, { recursive: true });
|
||||
fs.rmSync(path.join(testDir, '.codegraph'), { recursive: true, force: true });
|
||||
|
||||
cg = CodeGraph.initSync(testDir);
|
||||
await cg.indexAll();
|
||||
|
||||
const sidecar = path.join(testDir, 'explore-diag.jsonl');
|
||||
const previous = process.env.CODEGRAPH_EXPLORE_DEBUG;
|
||||
process.env.CODEGRAPH_EXPLORE_DEBUG = sidecar;
|
||||
try {
|
||||
response = (await new ToolHandler(cg).execute('codegraph_explore', { query: QUERY }))
|
||||
.content?.[0]?.text ?? '';
|
||||
} finally {
|
||||
if (previous === undefined) delete process.env.CODEGRAPH_EXPLORE_DEBUG;
|
||||
else process.env.CODEGRAPH_EXPLORE_DEBUG = previous;
|
||||
}
|
||||
const written = fs.readFileSync(sidecar, 'utf-8').trim().split('\n').filter(Boolean);
|
||||
report = JSON.parse(written[written.length - 1]!) as ExploreDiagnosticReport;
|
||||
|
||||
// A line counts as delivered only when the response numbers it AND the text
|
||||
// matches that source line — a line number quoted in prose must not count.
|
||||
sourceLines = fs.readFileSync(path.join(testDir, TARGET), 'utf-8').split('\n');
|
||||
delivered = new Set();
|
||||
for (const line of response.split('\n')) {
|
||||
const m = /^(\d+)\t(.*)$/.exec(line);
|
||||
if (!m) continue;
|
||||
const n = Number(m[1]);
|
||||
if (n >= 1 && n <= sourceLines.length && sourceLines[n - 1] === m[2]) delivered.add(n);
|
||||
}
|
||||
}, 120_000);
|
||||
|
||||
afterAll(() => {
|
||||
if (cg) cg.destroy();
|
||||
if (testDir && fs.existsSync(testDir)) fs.rmSync(testDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
/** The closures defined inside the factory, straight from the index. */
|
||||
const innerClosures = () => {
|
||||
const nodes = cg.getNodesInFile(TARGET);
|
||||
const factory = nodes.find((n) => n.name === FACTORY)!;
|
||||
return nodes.filter((n) => (n.kind === 'function' || n.kind === 'method')
|
||||
&& n.name !== FACTORY
|
||||
&& n.startLine > factory.startLine && n.endLine <= factory.endLine);
|
||||
};
|
||||
|
||||
describe('fixture shape — if this rots, the gate below means nothing', () => {
|
||||
it('holds one symbol spanning most of the file, with closures inside it', () => {
|
||||
const factory = cg.getNodesInFile(TARGET).find((n) => n.name === FACTORY);
|
||||
expect(factory, `${TARGET} has no ${FACTORY} node`).toBeDefined();
|
||||
// The envelope condition the >50% drop tests for — and `function`, the kind
|
||||
// that drop does not cover.
|
||||
expect(factory!.kind).toBe('function');
|
||||
expect(factory!.endLine - factory!.startLine + 1)
|
||||
.toBeGreaterThan(sourceLines.length * 0.5);
|
||||
expect(innerClosures().length).toBeGreaterThanOrEqual(8);
|
||||
});
|
||||
|
||||
it('is too long to ship whole, so it renders through the cluster path', () => {
|
||||
// Past WHOLE_FILE_MAX_LINES (220 for a non-central file): the whole-file
|
||||
// grace and buy arms cannot claim it, so the envelope actually matters.
|
||||
expect(sourceLines.length).toBeGreaterThan(220);
|
||||
expect(report.files.find((f) => f.path === TARGET)?.render).toBe('clusters');
|
||||
});
|
||||
});
|
||||
|
||||
describe('the gate', () => {
|
||||
it('delivers the closures the query named, not just the factory head', () => {
|
||||
const inner = innerClosures();
|
||||
for (const name of ['refreshMetrics', 'applyFilter']) {
|
||||
const node = inner.find((n) => n.name === name)!;
|
||||
expect(node, `${name} is not an inner closure any more`).toBeDefined();
|
||||
expect(delivered.has(node.startLine), `${name} definition line not delivered`).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it('delivers most of the closures, spread across the file', () => {
|
||||
const inner = innerClosures();
|
||||
const hit = inner.filter((n) => delivered.has(n.startLine));
|
||||
// Measured on the `feature/CG-24` tip: 7 of 11. The bar is half, so ordinary
|
||||
// budget movement does not fail the suite, but losing the closures does.
|
||||
expect(hit.length).toBeGreaterThanOrEqual(Math.ceil(inner.length / 2));
|
||||
// Not one contiguous head window off the top of the factory: the whole
|
||||
// point is that selection reaches symbols deep in the body.
|
||||
const last = inner[inner.length - 1]!;
|
||||
const deepest = Math.max(...hit.map((n) => n.startLine));
|
||||
expect(deepest).toBeGreaterThan((last.startLine + inner[0]!.startLine) / 2);
|
||||
});
|
||||
|
||||
it('never renders an empty section for the file', () => {
|
||||
const rec = report.files.find((f) => f.path === TARGET)!;
|
||||
expect(rec.emittedChars).toBeGreaterThan(0);
|
||||
expect(delivered.size).toBeGreaterThan(20);
|
||||
});
|
||||
|
||||
it('keeps the response inside the hard ceiling', () => {
|
||||
expect(report.envelope.chars).toBeLessThanOrEqual(report.budget.hardCeiling);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,199 @@
|
||||
/**
|
||||
* Standing gate for THE GUARANTEE (task CG-38): if the agent names a symbol and
|
||||
* that symbol's file is admitted to the response, the symbol's DEFINITION renders.
|
||||
*
|
||||
* This is the measurement the CG-24 epic never had. Its probes all score the
|
||||
* response in aggregate — envelope share, per-file spend, source totals, file
|
||||
* counts — and every one of them is green on a response that returns 25K of
|
||||
* source from the right file and still omits the function the agent asked for by
|
||||
* name. That is what CG-38 was: on a 1,414-line Svelte store, `queueMessage`
|
||||
* (L1087) and `flushQueuedMessages` (L1102) never rendered even though their file
|
||||
* won rank #1 with 67% of the envelope; the agent got the same-stem
|
||||
* `QueuedMessage` INTERFACE at L70 and had to Read the file to find the
|
||||
* functions. Longstanding, not an epic regression — the controlled bisect (index
|
||||
* held fixed, engine varied across every epic merge point) found it at every
|
||||
* build including pre-epic.
|
||||
*
|
||||
* Two independent causes, and the fixture below fails on either:
|
||||
*
|
||||
* 1. `buildFlowFromNamedSymbols` returned EMPTY — throwing away the NAMED-SYMBOL
|
||||
* IDENTITY along with the narrative — whenever the named symbols happened not
|
||||
* to form a call chain. Two sibling closures in one factory produce no chain,
|
||||
* no synthesized hop and no dispatch boundary, so both defs lost the
|
||||
* importance-9 rank that the named-def injection exists to give them.
|
||||
* 2. The ceiling trim cut in SOURCE ORDER, so whatever survived the shrink at
|
||||
* the END of a large file was always the first thing dropped.
|
||||
*
|
||||
* The fixture mirrors the reported file's geometry deliberately: a decoy
|
||||
* same-stem interface at L70, a factory closure at L104 spanning ~92% of the file
|
||||
* (so every symbol merges into ONE cluster), the target functions past L1000, and
|
||||
* a 2,500-line generated `.d.ts` for the ranker to penalise.
|
||||
*/
|
||||
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import CodeGraph from '../src/index';
|
||||
import { ToolHandler } from '../src/mcp/tools';
|
||||
|
||||
const FIXTURE = 'tail-render-ts';
|
||||
const TARGET = 'src/lib/session-store.ts';
|
||||
|
||||
let dir: string;
|
||||
let cg: CodeGraph;
|
||||
|
||||
/** Every `<n>\t<text>` line number the response actually sent. */
|
||||
function renderedLines(response: string): Set<number> {
|
||||
const out = new Set<number>();
|
||||
for (const m of response.matchAll(/^(\d+)\t/gm)) out.add(Number(m[1]));
|
||||
return out;
|
||||
}
|
||||
|
||||
async function explore(query: string): Promise<string> {
|
||||
const res = await new ToolHandler(cg).execute('codegraph_explore', { query });
|
||||
return res.content?.[0]?.text ?? '';
|
||||
}
|
||||
|
||||
function defLineOf(name: string): number {
|
||||
const node = cg.getNodesByName(name).find((n) => n.filePath === TARGET && n.startLine > 0);
|
||||
expect(node, `${name} is not indexed in ${TARGET}`).toBeDefined();
|
||||
return node!.startLine;
|
||||
}
|
||||
|
||||
beforeAll(async () => {
|
||||
dir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cg38-'));
|
||||
fs.cpSync(path.join(__dirname, 'fixtures', FIXTURE), dir, { recursive: true });
|
||||
fs.rmSync(path.join(dir, '.codegraph'), { recursive: true, force: true });
|
||||
cg = CodeGraph.initSync(dir);
|
||||
await cg.indexAll();
|
||||
}, 180_000);
|
||||
|
||||
afterAll(() => {
|
||||
cg?.destroy();
|
||||
if (dir && fs.existsSync(dir)) fs.rmSync(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
describe('CG-38 fixture shape — if this rots, the gate below means nothing', () => {
|
||||
it('puts the target functions past L1000 of a ~1,400-line file', () => {
|
||||
const lines = fs.readFileSync(path.join(dir, TARGET), 'utf-8').split('\n');
|
||||
expect(lines.length).toBeGreaterThan(1300);
|
||||
expect(defLineOf('queueMessage')).toBeGreaterThan(1000);
|
||||
expect(defLineOf('flushQueuedMessages')).toBeGreaterThan(1000);
|
||||
});
|
||||
|
||||
it('wraps them in a closure spanning most of the file, so they all cluster as one', () => {
|
||||
const lines = fs.readFileSync(path.join(dir, TARGET), 'utf-8').split('\n');
|
||||
const factory = cg.getNodesByName('createSessionStore')
|
||||
.find((n) => n.filePath === TARGET)!;
|
||||
expect(factory).toBeDefined();
|
||||
expect(factory.endLine - factory.startLine + 1).toBeGreaterThan(lines.length * 0.5);
|
||||
});
|
||||
|
||||
it('carries the same-stem decoy near the top', () => {
|
||||
const decoy = cg.getNodesByName('QueuedMessage').find((n) => n.filePath === TARGET)!;
|
||||
expect(decoy).toBeDefined();
|
||||
expect(decoy.kind).toBe('interface');
|
||||
expect(decoy.startLine).toBeLessThan(100);
|
||||
});
|
||||
|
||||
it('carries a generated declaration file for the ranker to penalise', () => {
|
||||
const dts = path.join(dir, 'types/worker-configuration.d.ts');
|
||||
expect(fs.existsSync(dts)).toBe(true);
|
||||
expect(fs.readFileSync(dts, 'utf-8').split('\n').length).toBeGreaterThan(2000);
|
||||
});
|
||||
|
||||
it('neither target calls the other — that absence is what produced no flow', () => {
|
||||
const queue = cg.getNodesByName('queueMessage').find((n) => n.filePath === TARGET)!;
|
||||
const flush = cg.getNodesByName('flushQueuedMessages').find((n) => n.filePath === TARGET)!;
|
||||
const between = [...cg.getCallees(queue.id), ...cg.getCallees(flush.id)]
|
||||
.filter(({ node }) => node.id === queue.id || node.id === flush.id);
|
||||
expect(between).toHaveLength(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('CG-38 — an agent-named symbol renders its definition', () => {
|
||||
/**
|
||||
* Both reported query shapes. They fail for different reasons — the symbol bag
|
||||
* never built a flow at all, the prose question built one and then lost the
|
||||
* tail to the ceiling trim — so a fix for one does not imply the other.
|
||||
*/
|
||||
const CASES: Array<{ shape: string; query: string; symbols: string[] }> = [
|
||||
{
|
||||
shape: 'symbol bag',
|
||||
query: 'queueMessage flushQueuedMessages',
|
||||
symbols: ['queueMessage', 'flushQueuedMessages'],
|
||||
},
|
||||
{
|
||||
shape: 'prose question',
|
||||
query: 'how does queueMessage hand its entries to flushQueuedMessages',
|
||||
symbols: ['queueMessage', 'flushQueuedMessages'],
|
||||
},
|
||||
{
|
||||
shape: 'three siblings, with the decoy interface competing',
|
||||
query: 'explain queueMessage, removeQueuedMessage and flushQueuedMessages',
|
||||
symbols: ['queueMessage', 'removeQueuedMessage', 'flushQueuedMessages'],
|
||||
},
|
||||
];
|
||||
|
||||
for (const { shape, query, symbols } of CASES) {
|
||||
it(`renders every named definition — ${shape}`, async () => {
|
||||
const response = await explore(query);
|
||||
const lines = renderedLines(response);
|
||||
for (const name of symbols) {
|
||||
const line = defLineOf(name);
|
||||
// The NAME alone proves nothing: it appears in the section header's
|
||||
// symbol list and at call sites whether or not the body was sent. Only
|
||||
// the definition LINE being among the rendered lines counts.
|
||||
expect(lines.has(line), `${name} (${TARGET}:${line}) did not render for "${query}"`)
|
||||
.toBe(true);
|
||||
}
|
||||
}, 120_000);
|
||||
}
|
||||
|
||||
it('never steers the agent to Read', async () => {
|
||||
const response = await explore('queueMessage flushQueuedMessages');
|
||||
expect(response).not.toMatch(/\buse Read\b|\bRead the file\b/i);
|
||||
}, 120_000);
|
||||
});
|
||||
|
||||
describe('CG-38 — a penalty on one file cannot shrink an unrelated file\'s render', () => {
|
||||
/**
|
||||
* The issue's sharpest lead: on an index where the generated `.d.ts` was NOT
|
||||
* flagged, the target file rendered ~581 lines including both symbols; on an
|
||||
* index where it WAS flagged, the same engine rendered 12. `rankPenalty` scales
|
||||
* `fileGraphScore`, which moves the relevance gate (6% of max) and so reshuffles
|
||||
* the admitted set — a demotion of one file must not cost an unrelated
|
||||
* top-ranked file its source.
|
||||
*
|
||||
* Flipping `files.generated` on that one row holds the INDEX constant and
|
||||
* attributes any delta to the ranker alone (the CG-25 method).
|
||||
*/
|
||||
const DTS = 'types/worker-configuration.d.ts';
|
||||
const QUERY = 'queueMessage flushQueuedMessages';
|
||||
|
||||
it('renders the same named definitions with the .d.ts flagged and unflagged', async () => {
|
||||
const setGenerated = (value: number) => {
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
const db = (cg as any).db?.getDatabase?.() ?? (cg as any).db?.db;
|
||||
db.prepare('UPDATE files SET generated = ? WHERE path = ?').run(value, DTS);
|
||||
};
|
||||
const linesFor = async () => renderedLines(await explore(QUERY));
|
||||
|
||||
const flagged = await linesFor();
|
||||
setGenerated(0);
|
||||
try {
|
||||
const unflagged = await linesFor();
|
||||
for (const name of ['queueMessage', 'flushQueuedMessages']) {
|
||||
const line = defLineOf(name);
|
||||
expect(flagged.has(line), `${name} missing with the .d.ts FLAGGED`).toBe(true);
|
||||
expect(unflagged.has(line), `${name} missing with the .d.ts UNFLAGGED`).toBe(true);
|
||||
}
|
||||
// The guarantee is about the named defs, not byte equality — the penalty is
|
||||
// supposed to move bytes around. What it must never do is cost the
|
||||
// top-ranked file the source the agent asked for.
|
||||
expect(unflagged.size).toBeGreaterThan(0);
|
||||
} finally {
|
||||
setGenerated(1);
|
||||
}
|
||||
}, 180_000);
|
||||
});
|
||||
@@ -0,0 +1,179 @@
|
||||
/**
|
||||
* Regression fixture for CG-30 — a cluster's top member may not overshoot the
|
||||
* file's budget without bound.
|
||||
*
|
||||
* `shrinkCluster` keeps the highest-importance member of an oversize cluster
|
||||
* WHOLE, deliberately: an empty file section sends the agent to Read, which is
|
||||
* the outcome explore exists to prevent. What it lacked was a bound. On the
|
||||
* originating repo one file emitted 22,376 chars against a 9,181-char
|
||||
* reservation — 2.44x — past both the per-file budget and the spine ceiling,
|
||||
* because its top member alone was that big. The overshoot is what collapses
|
||||
* `headroom` for every file ranked below it (CG-31), and it has a second face:
|
||||
* a member too big for the whole response ceiling makes the file drop out
|
||||
* entirely rather than render short.
|
||||
*
|
||||
* `__tests__/fixtures/oversize-member-ts/` reproduces both permanently. Three
|
||||
* report builders compete for one envelope, each a single long function far
|
||||
* bigger than any reservation it can earn beside its siblings. Measured against
|
||||
* the pre-fix build, this fixture produced:
|
||||
*
|
||||
* monthly.ts 12,391 chars emitted on a 3,334 budget (3.7x)
|
||||
* quarterly.ts dropped entirely — no headroom left (the CG-31 half)
|
||||
*
|
||||
* The gate below is that both are now bounded AND delivered: the bound cuts the
|
||||
* overshoot, and cutting the overshoot is what buys back the starved file.
|
||||
*
|
||||
* Measured against `spendable`, not `reserved`: the render paths bound
|
||||
* themselves by the reservation PLUS whatever slack the files above left on the
|
||||
* table, so a file legitimately spending inherited slack is not an overshoot.
|
||||
*/
|
||||
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import CodeGraph from '../src/index';
|
||||
import { ToolHandler } from '../src/mcp/tools';
|
||||
import { attributeSourceBytes } from '../src/mcp/explore-diagnostics';
|
||||
import type { ExploreDiagnosticReport, ExploreDiagnosticFile } from '../src/mcp/explore-diagnostics';
|
||||
|
||||
const FIXTURE_SRC = path.join(__dirname, 'fixtures', 'oversize-member-ts');
|
||||
|
||||
/** A symbol bag spanning the three builders — the sibling files compete. */
|
||||
const QUERY = 'buildMonthlyReport buildWeeklyReport buildQuarterlyReport formatReportRow persistReport';
|
||||
|
||||
/** The giant: one ~24K function, far past the whole-response ceiling. */
|
||||
const GIANT = 'src/report/monthly.ts';
|
||||
/** Mid-size: one ~11K function — the file the giant's overshoot used to starve. */
|
||||
const STARVED = 'src/report/quarterly.ts';
|
||||
|
||||
/** The bound: 1.5x, the same multiple the spine ceiling already draws. */
|
||||
const OVERSHOOT_FACTOR = 1.5;
|
||||
|
||||
describe('CG-30 — an oversize cluster member is bounded, not unbounded', () => {
|
||||
let testDir: string;
|
||||
let cg: CodeGraph;
|
||||
let response: string;
|
||||
let report: ExploreDiagnosticReport;
|
||||
let bytes: Map<string, number>;
|
||||
|
||||
const fileOf = (p: string): ExploreDiagnosticFile => {
|
||||
const rec = report.files.find((f) => f.path === p);
|
||||
if (!rec) throw new Error(`${p} absent from the diagnostic report`);
|
||||
return rec;
|
||||
};
|
||||
/** What the render paths actually bound themselves by. */
|
||||
const budgetOf = (rec: ExploreDiagnosticFile): number => rec.spendable ?? rec.allowance ?? 0;
|
||||
|
||||
beforeAll(async () => {
|
||||
testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cg30-'));
|
||||
fs.cpSync(FIXTURE_SRC, testDir, { recursive: true });
|
||||
fs.rmSync(path.join(testDir, '.codegraph'), { recursive: true, force: true });
|
||||
|
||||
cg = CodeGraph.initSync(testDir);
|
||||
await cg.indexAll();
|
||||
|
||||
// The per-file budget is only observable through the diagnostic sidecar, and
|
||||
// the whole gate is "emitted vs what the file was allowed to spend".
|
||||
const sidecar = path.join(testDir, 'explore-diag.jsonl');
|
||||
const previous = process.env.CODEGRAPH_EXPLORE_DEBUG;
|
||||
process.env.CODEGRAPH_EXPLORE_DEBUG = sidecar;
|
||||
try {
|
||||
const handler = new ToolHandler(cg);
|
||||
const result = await handler.execute('codegraph_explore', { query: QUERY });
|
||||
response = result.content?.[0]?.text ?? '';
|
||||
} finally {
|
||||
if (previous === undefined) delete process.env.CODEGRAPH_EXPLORE_DEBUG;
|
||||
else process.env.CODEGRAPH_EXPLORE_DEBUG = previous;
|
||||
}
|
||||
const written = fs.readFileSync(sidecar, 'utf-8').trim().split('\n').filter(Boolean);
|
||||
report = JSON.parse(written[written.length - 1]!) as ExploreDiagnosticReport;
|
||||
bytes = attributeSourceBytes(response);
|
||||
}, 120_000);
|
||||
|
||||
afterAll(() => {
|
||||
if (cg) cg.destroy();
|
||||
if (testDir && fs.existsSync(testDir)) fs.rmSync(testDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
// ── Fixture shape — if these rot, the gate below means nothing ─────────────
|
||||
|
||||
describe('fixture shape', () => {
|
||||
it('holds single members far bigger than any budget they can earn', () => {
|
||||
for (const file of [GIANT, STARVED]) {
|
||||
const source = fs.readFileSync(path.join(testDir, file), 'utf-8');
|
||||
const top = cg.getNodesInFile(file)
|
||||
.filter((n) => n.kind === 'function')
|
||||
.sort((a, b) => (b.endLine - b.startLine) - (a.endLine - a.startLine))[0];
|
||||
expect(top, `${file} has no function node`).toBeDefined();
|
||||
// One symbol, most of the file — the "top member alone is oversize" shape.
|
||||
expect(top!.endLine - top!.startLine).toBeGreaterThan(180);
|
||||
expect(source.length).toBeGreaterThan(budgetOf(fileOf(file)) * 2);
|
||||
}
|
||||
});
|
||||
|
||||
it('is too long to ship whole, so both render through the cluster path', () => {
|
||||
for (const file of [GIANT, STARVED]) {
|
||||
const lineCount = fs.readFileSync(path.join(testDir, file), 'utf-8').split('\n').length;
|
||||
// Past WHOLE_FILE_MAX_LINES (220 for a non-central file), so the
|
||||
// whole-file paths — grace and buy — cannot claim it.
|
||||
expect(lineCount, file).toBeGreaterThan(220);
|
||||
expect(fileOf(file).render, file).toBe('clusters');
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ── The gate ──────────────────────────────────────────────────────────────
|
||||
|
||||
describe('bounded overshoot', () => {
|
||||
it('CG-30 GATE: the giant no longer emits a multiple of its budget', () => {
|
||||
const rec = fileOf(GIANT);
|
||||
// Pre-fix this file emitted 12,391 on a 3,334 budget (3.7x).
|
||||
expect(rec.emittedChars).toBeLessThanOrEqual(
|
||||
Math.round(budgetOf(rec) * OVERSHOOT_FACTOR) + 1);
|
||||
});
|
||||
|
||||
it('CG-30 GATE: no clustered file emits past 1.5x what it may spend', () => {
|
||||
const over = report.files
|
||||
.filter((f) => f.render === 'clusters' && budgetOf(f) > 0)
|
||||
.filter((f) => f.emittedChars > Math.round(budgetOf(f) * OVERSHOOT_FACTOR) + 1)
|
||||
.map((f) => `${f.path}: ${f.emittedChars} of ${budgetOf(f)}`);
|
||||
expect(over).toEqual([]);
|
||||
});
|
||||
|
||||
it('CG-31: the file the overshoot used to starve is delivered', () => {
|
||||
// Pre-fix: dropped with skip reason `budget-clusters` — the giant above it
|
||||
// had already spent the headroom this file needed.
|
||||
expect(fileOf(STARVED).skipped).toBeNull();
|
||||
expect(bytes.get(STARVED) ?? 0).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('never emits an empty section — the invariant the old rule protected', () => {
|
||||
for (const rec of report.files) {
|
||||
if (rec.render !== 'clusters') continue;
|
||||
expect(rec.emittedChars, rec.path).toBeGreaterThan(0);
|
||||
}
|
||||
// And the windowed file still leads with the symbol the query named.
|
||||
expect(response).toContain('export function buildMonthlyReport');
|
||||
});
|
||||
|
||||
it('cuts on whole lines — a body is never sliced mid-line', () => {
|
||||
const source = fs.readFileSync(path.join(testDir, GIANT), 'utf-8').split('\n');
|
||||
const numbered = response
|
||||
.split('\n')
|
||||
.map((l) => /^(\d+)\t(.*)$/.exec(l))
|
||||
.filter((m): m is RegExpExecArray => m !== null)
|
||||
.filter((m) => Number(m[1]) >= 1 && Number(m[1]) <= source.length);
|
||||
const matching = numbered.filter((m) => source[Number(m[1]) - 1] === m[2]);
|
||||
// Every line the response numbers for this file is that whole source line.
|
||||
expect(matching.length).toBeGreaterThan(20);
|
||||
});
|
||||
|
||||
it('reports the cut rather than presenting a window as the whole file', () => {
|
||||
expect(fileOf(GIANT).clipped).toBe(true);
|
||||
});
|
||||
|
||||
it('keeps the response inside the hard ceiling', () => {
|
||||
expect(report.envelope.chars).toBeLessThanOrEqual(report.budget.hardCeiling);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,552 @@
|
||||
/**
|
||||
* Score-proportional byte allocation for codegraph_explore (CG-12 / #1500).
|
||||
*
|
||||
* `allocateExploreBudget` decides, before anything renders, how many chars of
|
||||
* source each ranked file may spend. Its contract is what stops the explore
|
||||
* envelope from following FILE SIZE — which is the bug #1500 reported: a small
|
||||
* weakly-relevant file shipped whole while the file that actually answered the
|
||||
* question was clipped at a flat per-file cap.
|
||||
*
|
||||
* These pin the allocator's invariants directly. End-to-end behaviour on the two
|
||||
* regression fixtures lives in `explore-allocation-1500.test.ts`.
|
||||
*/
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { allocateExploreBudget, getExploreOutputBudget, EXPLORE_ALLOCATION } from '../src/mcp/tools';
|
||||
import type { ExploreAllocationCandidate, ExploreAllocation, ExploreOutputBudget } from '../src/mcp/tools';
|
||||
|
||||
/** A candidate with sane defaults — tests override only what they're about. */
|
||||
const cand = (
|
||||
path: string,
|
||||
score: number,
|
||||
extra: Partial<ExploreAllocationCandidate> = {},
|
||||
): ExploreAllocationCandidate => ({ path, score, worth: 1, spine: false, ...extra });
|
||||
|
||||
const TIER_FILE_COUNTS = [10, 100, 300, 1000, 4000, 10000, 20000, 60000];
|
||||
|
||||
/**
|
||||
* The inline tool-result limit. Above it the host writes the response to a file
|
||||
* the agent Reads back, re-introducing the read this tool exists to prevent — so
|
||||
* it bounds every tier, not just the big ones (`hardCeiling`, tools.ts).
|
||||
*/
|
||||
const INLINE_CAP = 25000;
|
||||
|
||||
const reservedTotal = (a: ExploreAllocation) =>
|
||||
[...a.allowances.values()].reduce((sum, n) => sum + n, 0);
|
||||
|
||||
/**
|
||||
* What the render loop can actually emit for these reservations: each file's
|
||||
* slice, plus the whole-file grace it may overshoot by, plus the markdown
|
||||
* overhead charged per section. The allocator's job is to keep this inside the
|
||||
* envelope it was handed.
|
||||
*/
|
||||
const worstCaseEmission = (a: ExploreAllocation) => {
|
||||
let total = 0;
|
||||
for (const chars of a.allowances.values()) {
|
||||
total += chars + EXPLORE_ALLOCATION.FILE_OVERHEAD;
|
||||
}
|
||||
return total;
|
||||
};
|
||||
|
||||
describe('allocateExploreBudget — proportional split', () => {
|
||||
const budget = getExploreOutputBudget(1000); // 24,000 / 6,500 / 8 files
|
||||
|
||||
it('gives the higher-scoring file the bigger share', () => {
|
||||
const { allowances } = allocateExploreBudget(
|
||||
[cand('a.ts', 40), cand('b.ts', 10)],
|
||||
budget,
|
||||
8,
|
||||
);
|
||||
expect(allowances.get('a.ts')!).toBeGreaterThan(allowances.get('b.ts')!);
|
||||
});
|
||||
|
||||
it('scales the split with the score RATIO, not just the ordering', () => {
|
||||
// The heart of the fix. Under the old flat `maxCharsPerFile` both files got
|
||||
// the same cap and the split fell out of whichever happened to be small
|
||||
// enough to ship whole; here a 4x score buys materially more than a 1.1x one.
|
||||
const wide = allocateExploreBudget([cand('a.ts', 40), cand('b.ts', 10)], budget, 8).allowances;
|
||||
const narrow = allocateExploreBudget([cand('a.ts', 22), cand('b.ts', 20)], budget, 8).allowances;
|
||||
expect(wide.get('a.ts')! / wide.get('b.ts')!)
|
||||
.toBeGreaterThan(narrow.get('a.ts')! / narrow.get('b.ts')!);
|
||||
});
|
||||
|
||||
it('never reserves more than the envelope', () => {
|
||||
const { allowances, pool } = allocateExploreBudget(
|
||||
[cand('a.ts', 90), cand('b.ts', 40), cand('c.ts', 30), cand('d.ts', 12)],
|
||||
budget,
|
||||
8,
|
||||
);
|
||||
const reserved = [...allowances.values()].reduce((s, n) => s + n, 0);
|
||||
expect(reserved).toBeLessThanOrEqual(pool);
|
||||
expect(pool).toBeLessThanOrEqual(budget.maxOutputChars);
|
||||
});
|
||||
|
||||
it('caps any single file at the MAX_SHARE safety valve', () => {
|
||||
// The per-file cap is retired as the primary guard, but a lone dominant file
|
||||
// must still not be handed the entire response.
|
||||
const { allowances } = allocateExploreBudget([cand('god.ts', 500)], budget, 8);
|
||||
expect(allowances.get('god.ts')!).toBeLessThanOrEqual(Math.round(budget.maxOutputChars * 0.7));
|
||||
});
|
||||
|
||||
it('lets the top file exceed the old flat per-file cap when it earns it', () => {
|
||||
// The regression this task exists to fix: `maxCharsPerFile` clipped the file
|
||||
// that scored 4x its peers at exactly the same 6,500 as the noise.
|
||||
const { allowances } = allocateExploreBudget(
|
||||
[cand('answer.ts', 60), cand('noise.ts', 12)],
|
||||
budget,
|
||||
8,
|
||||
);
|
||||
expect(allowances.get('answer.ts')!).toBeGreaterThan(budget.maxCharsPerFile);
|
||||
});
|
||||
});
|
||||
|
||||
describe('allocateExploreBudget — the relative cliff', () => {
|
||||
const budget = getExploreOutputBudget(1000);
|
||||
|
||||
it('gives zero source to a file far below the top score', () => {
|
||||
const { allowances, cliffed } = allocateExploreBudget(
|
||||
[cand('answer.ts', 90), cand('incidental.ts', 3)],
|
||||
budget,
|
||||
8,
|
||||
);
|
||||
expect(cliffed).toContain('incidental.ts');
|
||||
expect(allowances.has('incidental.ts')).toBe(false);
|
||||
});
|
||||
|
||||
it('is RELATIVE — the same score survives against weaker company', () => {
|
||||
const strong = allocateExploreBudget([cand('a.ts', 90), cand('b.ts', 8)], budget, 8);
|
||||
const even = allocateExploreBudget([cand('a.ts', 12), cand('b.ts', 8)], budget, 8);
|
||||
expect(strong.cliffed).toContain('b.ts');
|
||||
expect(even.cliffed).not.toContain('b.ts');
|
||||
});
|
||||
|
||||
it('never rises above the score-floor ceiling, however dominant the top file', () => {
|
||||
// A 500-scoring god-file would otherwise put the cliff at 75 and silence
|
||||
// every peer the score floor had just deliberately admitted.
|
||||
const { cliffed } = allocateExploreBudget(
|
||||
[cand('god.ts', 500), cand('peer.ts', 13), cand('peer2.ts', 11)],
|
||||
budget,
|
||||
8,
|
||||
);
|
||||
expect(cliffed).toEqual([]);
|
||||
});
|
||||
|
||||
it('doubles the penalty on bytes that are worth less (generated / low-value)', () => {
|
||||
// `worth` is `rankPenalty` applied a second time: generated CRUD can rank on
|
||||
// name collisions while its bytes stay boilerplate. Same score, different fate.
|
||||
const { cliffed } = allocateExploreBudget(
|
||||
[cand('answer.ts', 60), cand('gen.ts', 12, { worth: 0.3 }), cand('hand.ts', 12)],
|
||||
budget,
|
||||
8,
|
||||
);
|
||||
expect(cliffed).toContain('gen.ts');
|
||||
expect(cliffed).not.toContain('hand.ts');
|
||||
});
|
||||
|
||||
it('exempts flow-spine files from the cliff', () => {
|
||||
// Clipping the spine causes the Read fallback — it IS the answer to a flow
|
||||
// question — so a spine file is never zeroed on relative score alone.
|
||||
const { allowances, cliffed } = allocateExploreBudget(
|
||||
[cand('a.ts', 400), cand('spine.ts', 2, { spine: true })],
|
||||
budget,
|
||||
8,
|
||||
);
|
||||
expect(cliffed).not.toContain('spine.ts');
|
||||
expect(allowances.get('spine.ts')!).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('never cliffs every candidate — an empty response costs a round-trip', () => {
|
||||
const { allowances, cliffed } = allocateExploreBudget([cand('only.ts', 0.5)], budget, 8);
|
||||
expect(cliffed).toEqual([]);
|
||||
expect(allowances.get('only.ts')!).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('hands a cliffed file\'s maxFiles slot to the next file down', () => {
|
||||
// The mechanism that got `BuildPayslip` into the #1500 response: cliffing is
|
||||
// not just "spend fewer bytes here", it frees the SLOT too.
|
||||
const { allowances } = allocateExploreBudget(
|
||||
[cand('a.ts', 90), cand('noise.ts', 2), cand('b.ts', 40)],
|
||||
budget,
|
||||
2,
|
||||
);
|
||||
expect([...allowances.keys()].sort()).toEqual(['a.ts', 'b.ts']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('allocateExploreBudget — the floor keeps diffuse questions useful', () => {
|
||||
const budget = getExploreOutputBudget(1000);
|
||||
|
||||
it('gives every admitted file a slice big enough for a method', () => {
|
||||
// A survey question must still return a spread. The earlier design cliffed a
|
||||
// starved file instead of flooring it, and that CASCADED: removing the
|
||||
// smallest raised everyone else so little that the next-smallest starved too,
|
||||
// eating six legitimately-ranked peers one at a time.
|
||||
const files = [cand('a.ts', 100), cand('b.ts', 90), ...Array.from({ length: 6 }, (_, i) => cand(`p${i}.ts`, 20))];
|
||||
const { allowances } = allocateExploreBudget(files, budget, 8);
|
||||
expect(allowances.size).toBe(8);
|
||||
for (const [, chars] of allowances) expect(chars).toBeGreaterThanOrEqual(700);
|
||||
});
|
||||
|
||||
it('serves fewer files well rather than many badly when the envelope cannot afford them', () => {
|
||||
const tiny = getExploreOutputBudget(10); // 13,000-char envelope
|
||||
const files = Array.from({ length: 40 }, (_, i) => cand(`f${i}.ts`, 50 - i * 0.1));
|
||||
const { allowances, cliffed } = allocateExploreBudget(files, tiny, 40);
|
||||
expect(allowances.size).toBeLessThan(40);
|
||||
expect(cliffed.length).toBeGreaterThan(0);
|
||||
for (const [, chars] of allowances) expect(chars).toBeGreaterThanOrEqual(700);
|
||||
const reserved = [...allowances.values()].reduce((s, n) => s + n, 0);
|
||||
expect(reserved).toBeLessThanOrEqual(tiny.maxOutputChars);
|
||||
});
|
||||
|
||||
it('returns an empty allocation for an empty candidate list', () => {
|
||||
const { allowances, cliffed } = allocateExploreBudget([], budget, 8);
|
||||
expect(allowances.size).toBe(0);
|
||||
expect(cliffed).toEqual([]);
|
||||
});
|
||||
|
||||
it('does not crash or over-allocate when every score is zero', () => {
|
||||
const { allowances } = allocateExploreBudget([cand('a.ts', 0), cand('b.ts', 0)], budget, 8);
|
||||
const reserved = [...allowances.values()].reduce((s, n) => s + n, 0);
|
||||
expect(reserved).toBeLessThanOrEqual(budget.maxOutputChars);
|
||||
});
|
||||
});
|
||||
|
||||
describe('allocateExploreBudget — tier invariant', () => {
|
||||
it('never gives a larger tier a smaller allowance than a smaller tier', () => {
|
||||
// The standing invariant from `getExploreOutputBudget`: a bigger project must
|
||||
// never be served LESS per file. It held for the flat cap by inspection; with
|
||||
// a proportional split it has to hold for the same candidate set across every
|
||||
// tier, which is what this walks.
|
||||
const files = [cand('a.ts', 60), cand('b.ts', 30), cand('c.ts', 15)];
|
||||
let previous: Map<string, number> | null = null;
|
||||
for (const fileCount of TIER_FILE_COUNTS) {
|
||||
const { allowances } = allocateExploreBudget(files, getExploreOutputBudget(fileCount), 8);
|
||||
if (previous) {
|
||||
for (const [path, chars] of allowances) {
|
||||
expect(chars, `${path} shrank at ${fileCount} files`).toBeGreaterThanOrEqual(previous.get(path)!);
|
||||
}
|
||||
}
|
||||
previous = allowances;
|
||||
}
|
||||
});
|
||||
|
||||
it('cliffs the same files at every tier — the cliff is relative, not sized', () => {
|
||||
const files = [cand('a.ts', 90), cand('noise.ts', 2)];
|
||||
const cliffs = TIER_FILE_COUNTS.map((n) =>
|
||||
allocateExploreBudget(files, getExploreOutputBudget(n), 8).cliffed.join(','));
|
||||
expect(new Set(cliffs).size).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
// ── CG-14 ───────────────────────────────────────────────────────────────────
|
||||
// Everything above pins the behaviours CG-12 was written to produce. What
|
||||
// follows pins the ones it must never produce: an over-spent envelope, a
|
||||
// starved diffuse query, a NaN slice — the failures that would ship silently
|
||||
// because they only surface as an agent falling back to Read.
|
||||
|
||||
describe('allocateExploreBudget — calibration', () => {
|
||||
it('pins the constants the two #1500 fixtures were calibrated against', () => {
|
||||
// Deliberately literal. Every other test here asserts an INVARIANT and reads
|
||||
// the constants, so it holds at any value; this one exists so that changing a
|
||||
// value is a visible decision rather than a silent re-tune of the fixtures.
|
||||
// If you change one, re-run `node scripts/agent-eval/probe-allocation.mjs`.
|
||||
expect(EXPLORE_ALLOCATION).toMatchObject({
|
||||
CLIFF_FRACTION: 0.15,
|
||||
CLIFF_MAX: 10,
|
||||
MIN_CHARS: 700,
|
||||
MAX_SHARE: 0.7,
|
||||
FILE_OVERHEAD: 200,
|
||||
SPINE_WEIGHT_BOOST: 2,
|
||||
WHOLE_FILE_GRACE_FRACTION: 0.15,
|
||||
WHOLE_FILE_GRACE_MAX: 800,
|
||||
});
|
||||
});
|
||||
|
||||
it('cliffs strictly BELOW the threshold, so a file exactly at it is still served', () => {
|
||||
// The boundary matters because `cliffAt` sits at CLIFF_MAX for any dominant
|
||||
// top file, which is also where the score floor's own ceiling sits — a file
|
||||
// that clears one must clear the other or the two gates disagree.
|
||||
const budget = getExploreOutputBudget(1000);
|
||||
const at = allocateExploreBudget([cand('top.ts', 1000), cand('probe.ts', 10)], budget, 8);
|
||||
const under = allocateExploreBudget([cand('top.ts', 1000), cand('probe.ts', 9.9)], budget, 8);
|
||||
expect(at.cliffAt).toBe(EXPLORE_ALLOCATION.CLIFF_MAX);
|
||||
expect(at.cliffed).not.toContain('probe.ts');
|
||||
expect(under.cliffed).toContain('probe.ts');
|
||||
});
|
||||
|
||||
it('tracks the top file until CLIFF_MAX caps it', () => {
|
||||
const budget = getExploreOutputBudget(1000);
|
||||
const cliffFor = (top: number) =>
|
||||
allocateExploreBudget([cand('top.ts', top), cand('b.ts', 1)], budget, 8).cliffAt;
|
||||
expect(cliffFor(20)).toBeCloseTo(20 * EXPLORE_ALLOCATION.CLIFF_FRACTION, 5);
|
||||
expect(cliffFor(50)).toBeCloseTo(50 * EXPLORE_ALLOCATION.CLIFF_FRACTION, 5);
|
||||
expect(cliffFor(500)).toBe(EXPLORE_ALLOCATION.CLIFF_MAX);
|
||||
});
|
||||
});
|
||||
|
||||
describe('allocateExploreBudget — envelope safety', () => {
|
||||
/** Deterministic LCG: a seeded sweep reproduces exactly, unlike Math.random. */
|
||||
const shapes = (): ExploreAllocationCandidate[][] => {
|
||||
let seed = 0x1500;
|
||||
const next = () => (seed = (seed * 1103515245 + 12345) & 0x7fffffff) / 0x7fffffff;
|
||||
const out: ExploreAllocationCandidate[][] = [];
|
||||
for (let n = 1; n <= 30; n++) {
|
||||
out.push(Array.from({ length: n }, (_, i) =>
|
||||
cand(`f${i}.ts`, Math.round(next() * 120 * 100) / 100, {
|
||||
worth: next() < 0.25 ? 0.3 : 1,
|
||||
spine: next() < 0.1,
|
||||
})));
|
||||
}
|
||||
return out;
|
||||
};
|
||||
|
||||
it('never reserves more than the envelope, at any tier or shape', () => {
|
||||
// The one invariant that must hold unconditionally: the render loop spends
|
||||
// reservations, so an over-allocation is an over-long response, and an
|
||||
// over-long response is externalized to a file the agent has to Read back.
|
||||
for (const fileCount of TIER_FILE_COUNTS) {
|
||||
const budget = getExploreOutputBudget(fileCount);
|
||||
for (const files of shapes()) {
|
||||
for (const maxFiles of [1, 4, 8, 30]) {
|
||||
const alloc = allocateExploreBudget(files, budget, maxFiles);
|
||||
const label = `${files.length} files, maxFiles=${maxFiles}, tier ${fileCount}`;
|
||||
expect(reservedTotal(alloc), label).toBeLessThanOrEqual(alloc.pool);
|
||||
expect(worstCaseEmission(alloc), label).toBeLessThanOrEqual(budget.maxOutputChars);
|
||||
for (const chars of alloc.allowances.values()) {
|
||||
expect(Number.isFinite(chars) && chars > 0, label).toBe(true);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it('never renders more files than maxFiles', () => {
|
||||
for (const maxFiles of [1, 2, 4, 8]) {
|
||||
const files = Array.from({ length: 25 }, (_, i) => cand(`f${i}.ts`, 100 - i));
|
||||
const alloc = allocateExploreBudget(files, getExploreOutputBudget(1000), maxFiles);
|
||||
expect(alloc.allowances.size).toBeLessThanOrEqual(maxFiles);
|
||||
}
|
||||
});
|
||||
|
||||
it('accounts for every candidate — a file is served, cliffed, or neither by choice', () => {
|
||||
// Nothing may vanish silently: a cliffed file is still NAMED in the response,
|
||||
// which is what makes withholding its bytes cheap. A file that is neither
|
||||
// served nor cliffed would be dropped without a pointer.
|
||||
const files = Array.from({ length: 25 }, (_, i) => cand(`f${i}.ts`, 100 - i * 4));
|
||||
const alloc = allocateExploreBudget(files, getExploreOutputBudget(1000), 8);
|
||||
const accounted = new Set([...alloc.allowances.keys(), ...alloc.cliffed]);
|
||||
expect(accounted.size).toBe(files.length);
|
||||
});
|
||||
|
||||
it('leaves the ~25K inline cap reachable only through the hard ceiling', () => {
|
||||
// Reservations always fit `maxOutputChars`, but the whole-file grace lets the
|
||||
// render loop overshoot a slice — so the envelope alone does NOT bound the
|
||||
// response, and `hardCeiling` is load-bearing rather than defensive. Pin both
|
||||
// halves: every tier's envelope is inside the inline cap, and the worst-case
|
||||
// graced emission is what the ceiling has to catch.
|
||||
for (const fileCount of TIER_FILE_COUNTS) {
|
||||
const budget = getExploreOutputBudget(fileCount);
|
||||
const hardCeiling = Math.min(Math.round(budget.maxOutputChars * 1.5), INLINE_CAP);
|
||||
expect(budget.maxOutputChars).toBeLessThan(INLINE_CAP);
|
||||
expect(hardCeiling).toBeLessThanOrEqual(INLINE_CAP);
|
||||
|
||||
const files = Array.from({ length: 8 }, (_, i) => cand(`f${i}.ts`, 90 - i * 9));
|
||||
const alloc = allocateExploreBudget(files, budget, 8);
|
||||
const graced = [...alloc.allowances.values()].reduce((sum, chars) => sum + chars
|
||||
+ Math.min(EXPLORE_ALLOCATION.WHOLE_FILE_GRACE_MAX,
|
||||
Math.round(chars * EXPLORE_ALLOCATION.WHOLE_FILE_GRACE_FRACTION))
|
||||
+ EXPLORE_ALLOCATION.FILE_OVERHEAD, 0);
|
||||
expect(graced).toBeGreaterThan(budget.maxOutputChars);
|
||||
expect(reservedTotal(alloc)).toBeLessThanOrEqual(budget.maxOutputChars);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('allocateExploreBudget — spine first', () => {
|
||||
const budget = getExploreOutputBudget(1000);
|
||||
|
||||
it('reserves more for a spine file than for an identically-scoring peer', () => {
|
||||
// "Spine first, unclipped" is enforced by WEIGHT, not by ordering: the spine
|
||||
// boost multiplies into the proportional split, so the flow gets its bytes
|
||||
// before any peripheral file competes for them.
|
||||
const { allowances } = allocateExploreBudget(
|
||||
[cand('peer.ts', 20), cand('spine.ts', 20, { spine: true })],
|
||||
budget,
|
||||
8,
|
||||
);
|
||||
expect(allowances.get('spine.ts')!).toBeGreaterThan(allowances.get('peer.ts')!);
|
||||
expect(allowances.get('spine.ts')! / allowances.get('peer.ts')!).toBeGreaterThan(1.3);
|
||||
});
|
||||
|
||||
it('keeps a spine file even when the envelope cannot afford everyone', () => {
|
||||
// The affordability trim keeps the highest weights and drops the rest in one
|
||||
// pass — but a dropped spine file breaks the flow, which is precisely the
|
||||
// failure that sends the agent back to Read. It is force-kept past the trim.
|
||||
const tiny = getExploreOutputBudget(10);
|
||||
const files = [
|
||||
...Array.from({ length: 20 }, (_, i) => cand(`f${i}.ts`, 100 - i)),
|
||||
cand('spine.ts', 4, { spine: true }),
|
||||
];
|
||||
const { allowances, cliffed } = allocateExploreBudget(files, tiny, 40);
|
||||
expect(allowances.has('spine.ts')).toBe(true);
|
||||
expect(cliffed).not.toContain('spine.ts');
|
||||
// Force-keeping it costs everyone a sliver — bounded, and the envelope still
|
||||
// holds. A real starvation regression would blow well past this.
|
||||
for (const [path, chars] of allowances) {
|
||||
expect(chars, path).toBeGreaterThanOrEqual(Math.round(EXPLORE_ALLOCATION.MIN_CHARS * 0.9));
|
||||
}
|
||||
expect(reservedTotal({ allowances, cliffed, cliffAt: 0, pool: 0 })).toBeLessThanOrEqual(tiny.maxOutputChars);
|
||||
});
|
||||
|
||||
it('does NOT exempt a spine file from maxFiles — the slot cap is separate', () => {
|
||||
// Documented boundary, not an oversight: the cliff is a relevance gate the
|
||||
// spine overrides, `maxFiles` is a response-shape cap it does not. In
|
||||
// practice the 2x boost lifts a spine file into the slots long before this
|
||||
// bites; the test exists so a future change to either gate is deliberate.
|
||||
const { allowances, cliffed } = allocateExploreBudget(
|
||||
[cand('a.ts', 90), cand('b.ts', 80), cand('spine.ts', 3, { spine: true })],
|
||||
budget,
|
||||
2,
|
||||
);
|
||||
expect(allowances.has('spine.ts')).toBe(false);
|
||||
expect(cliffed).toContain('spine.ts');
|
||||
});
|
||||
|
||||
it('serves a spine-only candidate set', () => {
|
||||
const { allowances, cliffed } = allocateExploreBudget(
|
||||
[cand('a.ts', 5, { spine: true }), cand('b.ts', 5, { spine: true })],
|
||||
budget,
|
||||
8,
|
||||
);
|
||||
expect(cliffed).toEqual([]);
|
||||
expect(allowances.size).toBe(2);
|
||||
});
|
||||
});
|
||||
|
||||
describe('allocateExploreBudget — degenerate inputs', () => {
|
||||
const budget = getExploreOutputBudget(1000);
|
||||
|
||||
it('splits evenly when every file scores identically, without starving any', () => {
|
||||
// The proportional split divides by the TOTAL weight, so an all-equal set is
|
||||
// the divide-by-a-degenerate-denominator case. Nobody is cliffed (nothing is
|
||||
// relatively weak) and everybody gets the same slice.
|
||||
for (const n of [2, 4, 8]) {
|
||||
const files = Array.from({ length: n }, (_, i) => cand(`f${i}.ts`, 17));
|
||||
const { allowances, cliffed } = allocateExploreBudget(files, budget, 8);
|
||||
expect(cliffed, `${n} files`).toEqual([]);
|
||||
expect(allowances.size, `${n} files`).toBe(n);
|
||||
const values = [...allowances.values()];
|
||||
expect(Math.max(...values) - Math.min(...values), `${n} files`).toBeLessThanOrEqual(1);
|
||||
for (const chars of values) expect(chars).toBeGreaterThanOrEqual(EXPLORE_ALLOCATION.MIN_CHARS);
|
||||
expect(worstCaseEmission({ allowances, cliffed, cliffAt: 0, pool: 0 }))
|
||||
.toBeLessThanOrEqual(budget.maxOutputChars);
|
||||
}
|
||||
});
|
||||
|
||||
it('gives a lone file a real answer, not the whole envelope', () => {
|
||||
const { allowances, cliffed } = allocateExploreBudget([cand('only.ts', 42)], budget, 8);
|
||||
expect(cliffed).toEqual([]);
|
||||
expect(allowances.size).toBe(1);
|
||||
const chars = allowances.get('only.ts')!;
|
||||
expect(chars).toBeGreaterThan(budget.maxCharsPerFile);
|
||||
expect(chars).toBe(Math.round(budget.maxOutputChars * EXPLORE_ALLOCATION.MAX_SHARE));
|
||||
});
|
||||
|
||||
it('holds a runaway top scorer to its share ceiling and still names the rest', () => {
|
||||
// One file 100x above everything else must not eat the response: the cliff
|
||||
// zeroes its peers' BYTES, but MAX_SHARE keeps the remainder for the pointer
|
||||
// list and the flow/relationship meta-text that lets the agent follow up.
|
||||
const { allowances, cliffed } = allocateExploreBudget(
|
||||
[cand('god.ts', 5000), cand('p1.ts', 9), cand('p2.ts', 8)],
|
||||
budget,
|
||||
8,
|
||||
);
|
||||
expect(allowances.get('god.ts')!).toBe(Math.round(budget.maxOutputChars * EXPLORE_ALLOCATION.MAX_SHARE));
|
||||
expect(reservedTotal({ allowances, cliffed, cliffAt: 0, pool: 0 }))
|
||||
.toBeLessThan(budget.maxOutputChars);
|
||||
expect(cliffed).toEqual(['p1.ts', 'p2.ts']);
|
||||
});
|
||||
|
||||
it('returns nothing to render when nothing scored', () => {
|
||||
for (const files of [
|
||||
[] as ExploreAllocationCandidate[],
|
||||
[cand('a.ts', 0), cand('b.ts', 0)],
|
||||
[cand('a.ts', 10, { worth: 0 }), cand('b.ts', 5, { worth: 0 })],
|
||||
[cand('a.ts', -5), cand('b.ts', -1)],
|
||||
]) {
|
||||
const { allowances, pool } = allocateExploreBudget(files, budget, 8);
|
||||
expect(allowances.size).toBe(0);
|
||||
expect(pool).toBeLessThanOrEqual(budget.maxOutputChars);
|
||||
}
|
||||
});
|
||||
|
||||
it('fails safe on a non-finite score instead of handing the render loop a NaN slice', () => {
|
||||
// Scores are finite sums in the pipeline, so this only has to not corrupt the
|
||||
// split — an Infinity weight would otherwise make every share Infinity/Infinity.
|
||||
for (const bad of [Infinity, NaN, -Infinity]) {
|
||||
const { allowances } = allocateExploreBudget([cand('bad.ts', bad), cand('ok.ts', 20)], budget, 8);
|
||||
for (const [path, chars] of allowances) {
|
||||
expect(Number.isFinite(chars), `${String(bad)} → ${path}`).toBe(true);
|
||||
}
|
||||
expect(allowances.get('ok.ts')).toBeGreaterThan(0);
|
||||
}
|
||||
});
|
||||
|
||||
it('renders nothing when maxFiles is zero, and still names every candidate', () => {
|
||||
const { allowances, cliffed } = allocateExploreBudget(
|
||||
[cand('a.ts', 10), cand('b.ts', 5)],
|
||||
budget,
|
||||
0,
|
||||
);
|
||||
expect(allowances.size).toBe(0);
|
||||
expect(cliffed).toEqual(['a.ts', 'b.ts']);
|
||||
});
|
||||
|
||||
it('survives an envelope too small for even one floored slice', () => {
|
||||
const cramped: ExploreOutputBudget = { ...budget, maxOutputChars: 300 };
|
||||
const { allowances, cliffed } = allocateExploreBudget(
|
||||
[cand('a.ts', 40), cand('b.ts', 30)],
|
||||
cramped,
|
||||
8,
|
||||
);
|
||||
expect(allowances.size).toBeLessThanOrEqual(1);
|
||||
for (const chars of allowances.values()) {
|
||||
expect(chars).toBeGreaterThan(0);
|
||||
expect(chars).toBeLessThanOrEqual(cramped.maxOutputChars);
|
||||
}
|
||||
expect([...allowances.keys(), ...cliffed].sort()).toEqual(['a.ts', 'b.ts']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('allocateExploreBudget — the diffuse-query control', () => {
|
||||
const budget = getExploreOutputBudget(1000);
|
||||
|
||||
it('keeps a survey-style spread readable — no file collapses to a fragment', () => {
|
||||
// The over-correction guard. Concentration is the point, but a genuinely
|
||||
// diffuse question (many comparably-relevant files) must still come back as a
|
||||
// usable spread: under-serving costs a whole round-trip, and the agent's
|
||||
// fallback is Grep, not a second explore.
|
||||
const files = Array.from({ length: 8 }, (_, i) => cand(`f${i}.ts`, 30 - i));
|
||||
const { allowances, cliffed } = allocateExploreBudget(files, budget, 8);
|
||||
expect(cliffed).toEqual([]);
|
||||
expect(allowances.size).toBe(8);
|
||||
const values = [...allowances.values()];
|
||||
for (const chars of values) expect(chars).toBeGreaterThanOrEqual(EXPLORE_ALLOCATION.MIN_CHARS);
|
||||
// Nobody is starved to make room for the leader: on a flat score curve the
|
||||
// spread between best and worst slice stays within a small multiple.
|
||||
expect(Math.max(...values) / Math.min(...values)).toBeLessThan(3);
|
||||
});
|
||||
|
||||
it('concentrates a precise query far harder than a diffuse one', () => {
|
||||
// Same envelope, same file count — only the score CURVE differs. This is the
|
||||
// whole thesis of the epic in one assertion.
|
||||
const topShareOf = (files: ExploreAllocationCandidate[]) => {
|
||||
const { allowances } = allocateExploreBudget(files, budget, 8);
|
||||
const values = [...allowances.values()];
|
||||
return Math.max(...values) / values.reduce((s, n) => s + n, 0);
|
||||
};
|
||||
const diffuse = topShareOf(Array.from({ length: 8 }, (_, i) => cand(`f${i}.ts`, 30 - i)));
|
||||
const precise = topShareOf([cand('answer.ts', 120), ...Array.from({ length: 7 }, (_, i) => cand(`f${i}.ts`, 14 - i))]);
|
||||
expect(diffuse).toBeLessThan(0.25);
|
||||
expect(precise).toBeGreaterThan(0.45);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,356 @@
|
||||
/**
|
||||
* Relevance scoring for `codegraph_explore` — CG-10 / #1500.
|
||||
*
|
||||
* The failure this pins: a file that merely NAME-COLLIDES with the query used to
|
||||
* score the same per match as the file that answers it, because every match in a
|
||||
* tier counted the same regardless of what was matched. Three
|
||||
* `scripts/agent-eval/*.mjs` harnesses took 63% of this repo's own "how does
|
||||
* explore allocate its output budget across files" response on nothing but a
|
||||
* local `const explore` and a `const BUDGET`.
|
||||
*
|
||||
* Four levers, one fixture family each:
|
||||
* 1. KIND WEIGHT — a match on a function/class outweighs one on a
|
||||
* variable/constant/parameter.
|
||||
* 2. ISOLATION — a weak-kind symbol nothing calls or references is a
|
||||
* pure collision and is demoted much harder.
|
||||
* 3. RELATIVE FLOOR — admission scales with the best file's score instead of
|
||||
* an absolute `>= 3`, capped so one direct match always
|
||||
* gets in and floored so a diffuse query keeps its spread.
|
||||
* 4. RANK PENALTY — generated and test/i18n files are discounted on BOTH
|
||||
* the score and the graph mass (the sort's primary key),
|
||||
* not merely tie-broken at equal score.
|
||||
*
|
||||
* Each fixture is a whole indexed project because the scoring reads the graph
|
||||
* (usage edges, RWR mass, the generated flag) — there is no seam to unit-test
|
||||
* the comparator against, and mocking one would pin the mock, not the behavior.
|
||||
*/
|
||||
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import CodeGraph from '../src/index';
|
||||
import { ToolHandler, RELEVANCE_KIND_WEIGHT } from '../src/mcp/tools';
|
||||
import { attributeSourceBytes } from '../src/mcp/explore-diagnostics';
|
||||
|
||||
/** Build + index a throwaway project from a `{ relPath: source }` map. */
|
||||
async function buildProject(
|
||||
prefix: string,
|
||||
files: Record<string, string>,
|
||||
): Promise<{ dir: string; cg: CodeGraph; handler: ToolHandler }> {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), prefix));
|
||||
for (const [rel, body] of Object.entries(files)) {
|
||||
const abs = path.join(dir, rel);
|
||||
fs.mkdirSync(path.dirname(abs), { recursive: true });
|
||||
fs.writeFileSync(abs, body.trimStart());
|
||||
}
|
||||
const cg = CodeGraph.initSync(dir);
|
||||
await cg.indexAll();
|
||||
return { dir, cg, handler: new ToolHandler(cg) };
|
||||
}
|
||||
|
||||
const cleanup = (dir: string, cg?: CodeGraph) => {
|
||||
if (cg) cg.destroy();
|
||||
if (dir && fs.existsSync(dir)) fs.rmSync(dir, { recursive: true, force: true });
|
||||
};
|
||||
|
||||
/**
|
||||
* Where a file's source section appears in the response. Sections are emitted in
|
||||
* final rank order, so this is the ranking assertion — which is what CG-10 owns.
|
||||
* How many BYTES each ranked file then gets is CG-12's (`maxCharsPerFile` and the
|
||||
* render loop still spend by file size, so a large low-ranked file can still
|
||||
* out-byte a small high-ranked one).
|
||||
*/
|
||||
const rankOf = (text: string, filePath: string): number => {
|
||||
const at = text.indexOf('**`' + filePath + '`**');
|
||||
if (at < 0) return Number.POSITIVE_INFINITY;
|
||||
return text.slice(0, at).split('**`').length;
|
||||
};
|
||||
|
||||
describe('RELEVANCE_KIND_WEIGHT', () => {
|
||||
it('ranks callables and types above members, and members above locals', () => {
|
||||
const callables = ['function', 'method', 'class', 'struct', 'interface', 'route', 'component'];
|
||||
for (const kind of callables) expect(RELEVANCE_KIND_WEIGHT[kind]).toBe(1);
|
||||
|
||||
for (const member of ['property', 'field', 'enum_member']) {
|
||||
expect(RELEVANCE_KIND_WEIGHT[member]!).toBeLessThan(RELEVANCE_KIND_WEIGHT.function!);
|
||||
expect(RELEVANCE_KIND_WEIGHT[member]!).toBeGreaterThan(RELEVANCE_KIND_WEIGHT.parameter!);
|
||||
}
|
||||
|
||||
// The #1500 kinds: incidental until the graph corroborates them.
|
||||
for (const weak of ['constant', 'variable', 'parameter']) {
|
||||
expect(RELEVANCE_KIND_WEIGHT[weak]!).toBeLessThan(0.5);
|
||||
}
|
||||
expect(RELEVANCE_KIND_WEIGHT.parameter!).toBeLessThan(RELEVANCE_KIND_WEIGHT.variable!);
|
||||
});
|
||||
});
|
||||
|
||||
describe('explore relevance scoring — incidental name collisions (#1500)', () => {
|
||||
let dir: string;
|
||||
let cg: CodeGraph;
|
||||
let handler: ToolHandler;
|
||||
|
||||
// Shape: one file DEFINES the dispatch mechanism; three unrelated scripts each
|
||||
// declare a lone unused `dispatch`/`registry` binding. Before CG-10 all four
|
||||
// cleared the floor and the three small scripts, shipping whole, took most of
|
||||
// the envelope from the large real file, which got clipped.
|
||||
beforeAll(async () => {
|
||||
const noise = (n: number) => `
|
||||
const dispatch = ${n};
|
||||
const registry = 'unused-${n}';
|
||||
|
||||
function unrelated${n}Helper(value) {
|
||||
return value + ${n};
|
||||
}
|
||||
`;
|
||||
({ dir, cg, handler } = await buildProject('codegraph-cg10-collide-', {
|
||||
'src/dispatcher.js': `
|
||||
import { lookupHandler } from './registry.js';
|
||||
|
||||
export function dispatch(event) {
|
||||
const handler = lookupHandler(event.type);
|
||||
if (!handler) return null;
|
||||
return runHandler(handler, event);
|
||||
}
|
||||
|
||||
export function runHandler(handler, event) {
|
||||
return handler(event.payload);
|
||||
}
|
||||
`,
|
||||
'src/registry.js': `
|
||||
const handlers = new Map();
|
||||
|
||||
export function registerHandler(type, fn) {
|
||||
handlers.set(type, fn);
|
||||
}
|
||||
|
||||
export function lookupHandler(type) {
|
||||
return handlers.get(type);
|
||||
}
|
||||
`,
|
||||
'scripts/report-a.js': noise(1),
|
||||
'scripts/report-b.js': noise(2),
|
||||
'scripts/report-c.js': noise(3),
|
||||
}));
|
||||
}, 120_000);
|
||||
|
||||
afterAll(() => cleanup(dir, cg));
|
||||
|
||||
const explore = async (query: string) => {
|
||||
const result = await handler.execute('codegraph_explore', { query });
|
||||
const text = result.content?.[0]?.text ?? '';
|
||||
return { text, bytes: attributeSourceBytes(text) };
|
||||
};
|
||||
|
||||
it('keeps files whose only match is an unused local out of the response', async () => {
|
||||
const { bytes } = await explore('how does dispatch route an event to its handler');
|
||||
for (const noiseFile of ['scripts/report-a.js', 'scripts/report-b.js', 'scripts/report-c.js']) {
|
||||
expect(bytes.get(noiseFile) ?? 0, `${noiseFile} must not reach the envelope`).toBe(0);
|
||||
}
|
||||
});
|
||||
|
||||
it('spends every delivered source byte on the files that define the mechanism', async () => {
|
||||
const { bytes } = await explore('how does dispatch route an event to its handler');
|
||||
let answer = 0;
|
||||
let noise = 0;
|
||||
for (const [file, n] of bytes) {
|
||||
if (file.startsWith('src/')) answer += n;
|
||||
else noise += n;
|
||||
}
|
||||
expect(answer).toBeGreaterThan(0);
|
||||
expect(noise).toBe(0);
|
||||
expect(bytes.get('src/dispatcher.js') ?? 0).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('still answers when the collision is the ONLY thing that matched', async () => {
|
||||
// Guard against over-correction: querying the noise term alone must not
|
||||
// produce an empty response. Under-serving costs the agent a round-trip, so
|
||||
// the floor's backfill has to keep the best of what matched.
|
||||
const { text } = await explore('unrelated2Helper');
|
||||
expect(text).not.toContain('No relevant code found');
|
||||
expect(text).toContain('unrelated2Helper');
|
||||
});
|
||||
});
|
||||
|
||||
describe('explore relevance scoring — generated source is penalized, not tie-broken', () => {
|
||||
let dir: string;
|
||||
let cg: CodeGraph;
|
||||
let handler: ToolHandler;
|
||||
|
||||
// The #1500 shape in miniature: the generated layer collides on every query
|
||||
// term AND carries more call-graph mass than the hand-written use-case, so a
|
||||
// generated-as-tiebreak-only rule leaves it ranked first.
|
||||
beforeAll(async () => {
|
||||
({ dir, cg, handler } = await buildProject('codegraph-cg10-generated-', {
|
||||
'go.mod': 'module example.com/billing\n\ngo 1.22\n',
|
||||
'internal/usecase/billing/invoice.go': `
|
||||
package billing
|
||||
|
||||
// Service runs the month-end invoicing workflow.
|
||||
type Service struct {
|
||||
store Store
|
||||
}
|
||||
|
||||
// RunInvoiceCycle is the hand-written business rule the question is about.
|
||||
func (s *Service) RunInvoiceCycle(month string) error {
|
||||
lines := s.CollectInvoiceLines(month)
|
||||
total := s.CalculateInvoiceTotal(lines)
|
||||
return s.store.Save(month, total)
|
||||
}
|
||||
|
||||
func (s *Service) CollectInvoiceLines(month string) []int {
|
||||
return []int{1, 2, 3}
|
||||
}
|
||||
|
||||
func (s *Service) CalculateInvoiceTotal(lines []int) int {
|
||||
sum := 0
|
||||
for _, l := range lines {
|
||||
sum += l
|
||||
}
|
||||
return sum
|
||||
}
|
||||
`,
|
||||
'internal/usecase/billing/store.go': `
|
||||
package billing
|
||||
|
||||
type Store interface {
|
||||
Save(month string, total int) error
|
||||
}
|
||||
`,
|
||||
// Ordinary filename — ONLY the content banner betrays it (the CG-5 case).
|
||||
'internal/gen/billing/invoice.go': `
|
||||
// Code generated by billingkit. DO NOT EDIT.
|
||||
|
||||
package gen
|
||||
|
||||
type InvoiceRow struct {
|
||||
Month string
|
||||
Total int
|
||||
}
|
||||
|
||||
type InvoiceCreateRequest struct {
|
||||
Month string
|
||||
}
|
||||
|
||||
func CreateInvoice(req InvoiceCreateRequest) InvoiceRow {
|
||||
return BuildInvoice(req.Month, 0)
|
||||
}
|
||||
|
||||
func BuildInvoice(month string, total int) InvoiceRow {
|
||||
return InvoiceRow{Month: month, Total: total}
|
||||
}
|
||||
|
||||
func CalculateInvoiceTotal(rows []InvoiceRow) int {
|
||||
sum := 0
|
||||
for _, r := range rows {
|
||||
sum += r.Total
|
||||
}
|
||||
return sum
|
||||
}
|
||||
|
||||
func ListInvoices(month string) []InvoiceRow {
|
||||
return []InvoiceRow{BuildInvoice(month, 0)}
|
||||
}
|
||||
|
||||
func CollectInvoiceLines(month string) []InvoiceRow {
|
||||
return ListInvoices(month)
|
||||
}
|
||||
|
||||
func RunInvoiceCycle(month string) InvoiceRow {
|
||||
rows := CollectInvoiceLines(month)
|
||||
return BuildInvoice(month, CalculateInvoiceTotal(rows))
|
||||
}
|
||||
`,
|
||||
}));
|
||||
}, 120_000);
|
||||
|
||||
afterAll(() => cleanup(dir, cg));
|
||||
|
||||
it('indexes the ordinary-named generated file via its content banner', () => {
|
||||
expect(cg.getFile('internal/gen/billing/invoice.go')?.generated).toBe(true);
|
||||
expect(cg.getFile('internal/usecase/billing/invoice.go')?.generated).toBe(false);
|
||||
});
|
||||
|
||||
it('ranks the hand-written workflow above its generated twin', async () => {
|
||||
const result = await handler.execute('codegraph_explore', {
|
||||
query: 'how does the invoice cycle collect lines and calculate the total',
|
||||
});
|
||||
const text = result.content?.[0]?.text ?? '';
|
||||
|
||||
// The generated file collides on EVERY query term and carries call-graph
|
||||
// mass of its own, so with generated status as a mere tiebreak-at-equal-score
|
||||
// it ranked first. The penalty scales its score AND its graph mass, which is
|
||||
// the key the comparator actually sorts on.
|
||||
const handWritten = rankOf(text, 'internal/usecase/billing/invoice.go');
|
||||
const generated = rankOf(text, 'internal/gen/billing/invoice.go');
|
||||
expect(handWritten).toBeLessThan(generated);
|
||||
expect(attributeSourceBytes(text).get('internal/usecase/billing/invoice.go') ?? 0)
|
||||
.toBeGreaterThan(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('explore relevance scoring — test files never buy the envelope', () => {
|
||||
let dir: string;
|
||||
let cg: CodeGraph;
|
||||
let handler: ToolHandler;
|
||||
|
||||
// A repo-ROOT `test/` directory — the shape express and most of npm/Go use.
|
||||
// The old detector anchored on a leading `/`, so `test/x.js` never matched it
|
||||
// and express's routing question spent 59% of its envelope on three test files.
|
||||
beforeAll(async () => {
|
||||
const spec = (n: number) => `
|
||||
const { parseRoute } = require('../lib/router.js');
|
||||
|
||||
describe('parseRoute ${n}', () => {
|
||||
it('parses a route ${n}', () => {
|
||||
parseRoute('/a/${n}');
|
||||
});
|
||||
it('parses another route ${n}', () => {
|
||||
parseRoute('/b/${n}');
|
||||
});
|
||||
});
|
||||
`;
|
||||
({ dir, cg, handler } = await buildProject('codegraph-cg10-lowvalue-', {
|
||||
'lib/router.js': `
|
||||
exports.parseRoute = function parseRoute(pathname) {
|
||||
const segments = pathname.split('/').filter(Boolean);
|
||||
return { segments, matched: matchRoute(segments) };
|
||||
};
|
||||
|
||||
function matchRoute(segments) {
|
||||
return segments.length > 0;
|
||||
}
|
||||
`,
|
||||
'lib/dispatch.js': `
|
||||
const { parseRoute } = require('./router.js');
|
||||
|
||||
exports.dispatchRoute = function dispatchRoute(pathname) {
|
||||
return parseRoute(pathname);
|
||||
};
|
||||
`,
|
||||
'test/router.raw.js': spec(1),
|
||||
'test/router.json.js': spec(2),
|
||||
'test/router.text.js': spec(3),
|
||||
}));
|
||||
}, 120_000);
|
||||
|
||||
afterAll(() => cleanup(dir, cg));
|
||||
|
||||
it('excludes a repo-root test/ directory from the envelope', async () => {
|
||||
const result = await handler.execute('codegraph_explore', {
|
||||
query: 'how does the router parse and dispatch a route',
|
||||
});
|
||||
const bytes = attributeSourceBytes(result.content?.[0]?.text ?? '');
|
||||
for (const [file, n] of bytes) {
|
||||
expect(n === 0 || !file.startsWith('test/'), `${file} took ${n} chars`).toBe(true);
|
||||
}
|
||||
expect(bytes.get('lib/router.js') ?? 0).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('still returns tests when the query is about them', async () => {
|
||||
const result = await handler.execute('codegraph_explore', {
|
||||
query: 'which tests cover parseRoute',
|
||||
});
|
||||
const text = result.content?.[0]?.text ?? '';
|
||||
expect(text).not.toContain('No relevant code found');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,265 @@
|
||||
/**
|
||||
* Regression fixture for CG-26 — the end-to-end reservation invariant.
|
||||
*
|
||||
* Every admitted file receives at least its reservation before any file draws
|
||||
* on carry-forward slack.
|
||||
*
|
||||
* CG-30 bounded how far an oversize cluster member may overshoot and CG-31 gave
|
||||
* the cluster path a displacement guard. This pins the invariant they jointly
|
||||
* satisfy across EVERY render path — cluster, whole-file grace, whole-file BUY —
|
||||
* and in BOTH directions: the top-ranked file when the files below it overspend,
|
||||
* and an admitted lower-ranked file when the top one does.
|
||||
*
|
||||
* Two things CG-26 fixed are pinned here because nothing else can see them:
|
||||
*
|
||||
* - The whole-file arms were fit-tested against raw room before the ceiling,
|
||||
* never against what was still owed below. A grace-sized file could take a
|
||||
* pending file's reservation on its way to the ceiling; okhttp's
|
||||
* `CallServerInterceptor.kt` shipped 8,499 chars on a 5,964 funded ceiling
|
||||
* and the rank-6 file below it delivered nothing.
|
||||
* - Every section was charged a flat 200 chars of overhead while a real header
|
||||
* runs 300–500. The loop believed it had room it did not have (okhttp
|
||||
* rendered 26,601 chars against a 24,400 ceiling), so the final truncation
|
||||
* threw a fully-rendered section away — the same starvation, arriving after
|
||||
* the guard had done its work.
|
||||
*
|
||||
* Shares the `displacement-ts` fixture: four pipeline stages competing for one
|
||||
* envelope, the first a single ~20K function, padded past 500 indexed files so
|
||||
* the response sits on the 24K tier where reservations genuinely saturate the
|
||||
* ceiling.
|
||||
*/
|
||||
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import CodeGraph from '../src/index';
|
||||
import { ToolHandler } from '../src/mcp/tools';
|
||||
import { attributeSourceBytes } from '../src/mcp/explore-diagnostics';
|
||||
import type { ExploreDiagnosticReport, ExploreDiagnosticFile } from '../src/mcp/explore-diagnostics';
|
||||
|
||||
const FIXTURE_SRC = path.join(__dirname, 'fixtures', 'displacement-ts');
|
||||
const FILLER_FILES = 520;
|
||||
|
||||
/** The giant: one ~20K function. Ranks #1 under the spread query. */
|
||||
const GIANT = 'src/pipeline/ingest.ts';
|
||||
|
||||
/**
|
||||
* Three shapes, so the invariant is tested from both sides:
|
||||
* spread — every stage named; the giant ranks #1 and overspends downwards.
|
||||
* tail — the stages BELOW the giant named; something small ranks #1 while
|
||||
* the giant competes from underneath. This is the direction CG-31's
|
||||
* fixture could not reach.
|
||||
* precise — one symbol. The concentration case the guard must not flatten.
|
||||
*/
|
||||
const QUERIES = {
|
||||
spread: 'ingestRecords normalizeRecords enrichRecords publishRecords',
|
||||
tail: 'publishRecords sinkRecord PipelineRecord ingestRecords',
|
||||
precise: 'ingestRecords',
|
||||
} as const;
|
||||
type Shape = keyof typeof QUERIES;
|
||||
|
||||
interface Probe {
|
||||
response: string;
|
||||
report: ExploreDiagnosticReport;
|
||||
bytes: Map<string, number>;
|
||||
}
|
||||
|
||||
describe('CG-26 — no admitted file is starved, on any render path', () => {
|
||||
let testDir: string;
|
||||
let cg: CodeGraph;
|
||||
const probes = {} as Record<Shape, Probe>;
|
||||
|
||||
/** Admitted = the allocator reserved bytes for it. */
|
||||
const admitted = (probe: Probe): ExploreDiagnosticFile[] =>
|
||||
probe.report.files.filter((f) => (f.allowance ?? 0) > 0);
|
||||
const all = (): Probe[] => Object.values(probes);
|
||||
|
||||
beforeAll(async () => {
|
||||
testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cg26-'));
|
||||
fs.cpSync(FIXTURE_SRC, testDir, { recursive: true });
|
||||
fs.rmSync(path.join(testDir, '.codegraph'), { recursive: true, force: true });
|
||||
|
||||
const filler = path.join(testDir, 'src', 'generated');
|
||||
fs.mkdirSync(filler, { recursive: true });
|
||||
for (let i = 0; i < FILLER_FILES; i++) {
|
||||
fs.writeFileSync(
|
||||
path.join(filler, `unit${i}.ts`),
|
||||
`export const seed${i} = ${i};\n`
|
||||
+ `export function widget${i}(n: number): number {\n return n * ${i + 1} + seed${i};\n}\n`,
|
||||
);
|
||||
}
|
||||
|
||||
cg = CodeGraph.initSync(testDir);
|
||||
await cg.indexAll();
|
||||
|
||||
const sidecar = path.join(testDir, 'explore-diag.jsonl');
|
||||
const previous = process.env.CODEGRAPH_EXPLORE_DEBUG;
|
||||
process.env.CODEGRAPH_EXPLORE_DEBUG = sidecar;
|
||||
try {
|
||||
const handler = new ToolHandler(cg);
|
||||
for (const [shape, query] of Object.entries(QUERIES) as [Shape, string][]) {
|
||||
const result = await handler.execute('codegraph_explore', { query });
|
||||
const response = result.content?.[0]?.text ?? '';
|
||||
const written = fs.readFileSync(sidecar, 'utf-8').trim().split('\n').filter(Boolean);
|
||||
probes[shape] = {
|
||||
response,
|
||||
report: JSON.parse(written[written.length - 1]!) as ExploreDiagnosticReport,
|
||||
bytes: attributeSourceBytes(response),
|
||||
};
|
||||
}
|
||||
} finally {
|
||||
if (previous === undefined) delete process.env.CODEGRAPH_EXPLORE_DEBUG;
|
||||
else process.env.CODEGRAPH_EXPLORE_DEBUG = previous;
|
||||
}
|
||||
}, 180_000);
|
||||
|
||||
afterAll(() => {
|
||||
if (cg) cg.destroy();
|
||||
if (testDir && fs.existsSync(testDir)) fs.rmSync(testDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
// ── Fixture shape — if these rot, the gates below mean nothing ─────────────
|
||||
|
||||
describe('fixture shape', () => {
|
||||
it('sits on the 24K tier, where the reservations saturate the ceiling', () => {
|
||||
expect(cg.getStats().fileCount).toBeGreaterThanOrEqual(500);
|
||||
for (const probe of all()) expect(probe.report.budget.maxOutputChars).toBe(24000);
|
||||
});
|
||||
|
||||
it('exercises both directions — the giant ranks #1 in one shape and lower in another', () => {
|
||||
// Which shape puts it where is the ranker's business and may move; that
|
||||
// it lands on BOTH sides across the three is what makes the gates below
|
||||
// test the invariant rather than one arrangement of it.
|
||||
const ranks = all().map((p) => p.report.files.find((f) => f.path === GIANT)?.rank ?? -1);
|
||||
expect(ranks).toContain(1);
|
||||
expect(ranks.some((r) => r > 1)).toBe(true);
|
||||
});
|
||||
|
||||
it('exercises both render paths — something ships whole, something clusters', () => {
|
||||
const modes = new Set(all().flatMap((p) => p.report.files.map((f) => f.render)));
|
||||
expect(modes).toContain('clusters');
|
||||
expect(modes).toContain('whole');
|
||||
});
|
||||
});
|
||||
|
||||
// ── The invariant ─────────────────────────────────────────────────────────
|
||||
|
||||
describe('the reservation invariant', () => {
|
||||
it('CG-26 GATE: no file on ANY render path emits past what was still free', () => {
|
||||
// CG-31 pinned this for `clusters` only. The whole-file arms were fit-
|
||||
// tested against `renderCeiling - totalChars`, which is everyone's room,
|
||||
// not this file's — so a whole render could spend a reservation the loop
|
||||
// had already promised further down.
|
||||
for (const [shape, probe] of Object.entries(probes) as [Shape, Probe][]) {
|
||||
const over = probe.report.files
|
||||
.filter((f) => f.render !== null && f.render !== 'dropped' && f.funded !== null)
|
||||
// +1 for the render loop's own rounding on a windowed cut.
|
||||
.filter((f) => f.emittedChars > f.funded! + 1)
|
||||
.map((f) => `${shape}/${f.path}: ${f.emittedChars} emitted of ${f.funded} funded (${f.render})`);
|
||||
expect(over).toEqual([]);
|
||||
}
|
||||
});
|
||||
|
||||
it('CG-26 GATE: every admitted file is delivered, whatever its rank', () => {
|
||||
for (const [shape, probe] of Object.entries(probes) as [Shape, Probe][]) {
|
||||
for (const rec of admitted(probe)) {
|
||||
expect(rec.skipped, `${shape}/${rec.path} skipped`).toBeNull();
|
||||
expect(probe.bytes.get(rec.path) ?? 0, `${shape}/${rec.path} bytes`).toBeGreaterThan(0);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it('CG-26 GATE: the rank-#1 file gets its reservation even when a file below overspends', () => {
|
||||
// The direction CG-31's fixture could not reach: under `tail` the giant
|
||||
// ranks below a small file and draws far past its own reservation from
|
||||
// carry-forward slack. Rank #1 must still receive what it was promised
|
||||
// (or its whole file, if that is less).
|
||||
for (const [shape, probe] of Object.entries(probes) as [Shape, Probe][]) {
|
||||
const top = admitted(probe).sort((a, b) => a.rank - b.rank)[0];
|
||||
if (!top) continue;
|
||||
const onDisk = fs.statSync(path.join(testDir, top.path)).size;
|
||||
expect(probe.bytes.get(top.path) ?? 0, `${shape}/${top.path}`)
|
||||
.toBeGreaterThanOrEqual(Math.min(top.allowance!, onDisk) * 0.9);
|
||||
}
|
||||
});
|
||||
|
||||
it('and the gate above is not vacuous — a lower-ranked file does overspend', () => {
|
||||
const overspenders = (probe: Probe) => admitted(probe)
|
||||
.filter((f) => f.rank > 1 && f.emittedChars > f.allowance!);
|
||||
expect(overspenders(probes.tail).length).toBeGreaterThan(0);
|
||||
});
|
||||
});
|
||||
|
||||
// ── What the ceiling must no longer do ────────────────────────────────────
|
||||
|
||||
describe('the hard ceiling never throws a rendered section away', () => {
|
||||
it('the render loop spends what it counts — nothing is allocated past the ceiling', () => {
|
||||
// Sections used to be charged a flat 200 chars against a header that runs
|
||||
// 300–500, so the loop over-filled and the final truncation dropped whole
|
||||
// sections. `allocatedChars` is the pre-truncation length: it staying
|
||||
// under the ceiling IS the accounting being exact.
|
||||
for (const [shape, probe] of Object.entries(probes) as [Shape, Probe][]) {
|
||||
expect(probe.report.envelope.allocatedChars, shape)
|
||||
.toBeLessThanOrEqual(probe.report.budget.hardCeiling);
|
||||
expect(probe.report.envelope.truncated, shape).toBe(false);
|
||||
}
|
||||
});
|
||||
|
||||
it('no file is rendered and then dropped', () => {
|
||||
for (const probe of all()) {
|
||||
expect(probe.report.files.filter((f) => f.render === 'dropped')).toEqual([]);
|
||||
}
|
||||
});
|
||||
|
||||
it('keeps the response inside the hard ceiling', () => {
|
||||
for (const probe of all()) {
|
||||
expect(probe.report.envelope.chars).toBeLessThanOrEqual(probe.report.budget.hardCeiling);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ── The epilogue is budgeted, not discarded ───────────────────────────────
|
||||
|
||||
describe('the epilogue the loop budgeted for is the epilogue it emits', () => {
|
||||
it('a response that withheld files still says so, and says to explore not Read', () => {
|
||||
// The flat 600-char margin was neither the epilogue's size nor a bound on
|
||||
// it, so a saturated response shipped with no pointer list and no
|
||||
// reminders at all. Whatever else is traded away, the agent must be told
|
||||
// an uncovered area exists and that another explore reaches it.
|
||||
for (const [shape, probe] of Object.entries(probes) as [Shape, Probe][]) {
|
||||
const withheld = probe.report.files.some(
|
||||
(f) => f.render === null || (probe.bytes.get(f.path) ?? 0) === 0);
|
||||
if (!withheld) continue;
|
||||
expect(
|
||||
/Not shown above|omitted for size|codegraph_explore/.test(probe.response),
|
||||
`${shape} withheld files without saying where to look`,
|
||||
).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it('never steers the agent to Read', () => {
|
||||
for (const probe of all()) {
|
||||
expect(/use (the )?Read|fall back to Read(?!ing those files)/i.test(probe.response)).toBe(false);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ── The thing the invariant must NOT become ───────────────────────────────
|
||||
|
||||
describe('concentration survives', () => {
|
||||
it('a precise symbol query still puts the most source in the named file', () => {
|
||||
const mine = probes.precise.bytes.get(GIANT) ?? 0;
|
||||
expect(mine).toBeGreaterThan(0);
|
||||
for (const [p, n] of probes.precise.bytes) {
|
||||
if (p === GIANT) continue;
|
||||
expect(mine, `${GIANT} vs ${p}`).toBeGreaterThan(n);
|
||||
}
|
||||
});
|
||||
|
||||
it('is not an even split — the named file outspends its equal share', () => {
|
||||
const rec = probes.precise.report.files.find((f) => f.path === GIANT)!;
|
||||
const even = probes.precise.report.budget.maxOutputChars / admitted(probes.precise).length;
|
||||
expect(rec.emittedChars).toBeGreaterThan(even);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,469 @@
|
||||
/**
|
||||
* Session-scoped explore call state (CG-17).
|
||||
*
|
||||
* The tracker is the foundation for cross-call dedup (CG-18) and budget decay
|
||||
* (CG-19), so what it must get right is what those two will trust: the count of
|
||||
* calls, the line ranges already served, and — above all — WHOSE they are. Two
|
||||
* agents on one daemon share a ToolHandler and a worker pool; if their histories
|
||||
* blend, a dedup built on this would withhold source from an agent that never
|
||||
* saw it, and the agent Reads the file. That is the failure this suite guards.
|
||||
*
|
||||
* Three layers:
|
||||
* 1. the state container itself — keying, monotonic call index, bounds;
|
||||
* 2. the handler seam — a real explore against a real index records real
|
||||
* ranges, and the emission side-channel NEVER reaches the response;
|
||||
* 3. the session seam — separate sessions on one engine, separate state.
|
||||
*/
|
||||
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import CodeGraph from '../src/index';
|
||||
import { ToolHandler } from '../src/mcp/tools';
|
||||
import { MCPSession } from '../src/mcp/session';
|
||||
import type { MCPEngine } from '../src/mcp/engine';
|
||||
import type { JsonRpcTransport, JsonRpcRequest, JsonRpcNotification } from '../src/mcp/transport';
|
||||
import {
|
||||
EXPLORE_EMISSION_KEY,
|
||||
EXPLORE_SESSION_LIMITS,
|
||||
EXPLORE_SESSION_VIEW_ARG,
|
||||
ExploreSessionState,
|
||||
coalesceRanges,
|
||||
exploreProjectKey,
|
||||
rangesCover,
|
||||
readExploreSessionView,
|
||||
viewForProject,
|
||||
type ExploreEmission,
|
||||
} from '../src/mcp/explore-session-state';
|
||||
|
||||
const FIXTURE_SRC = path.join(__dirname, 'fixtures', 'payroll-go');
|
||||
const QUERY = 'how does payroll cycle create and calculate payslips?';
|
||||
|
||||
/** An emission shaped like a real one, for the container-level tests. */
|
||||
function emission(root: string, over: Partial<ExploreEmission> = {}): ExploreEmission {
|
||||
return {
|
||||
projectRoot: root,
|
||||
query: 'q',
|
||||
files: [{ path: 'a.ts', ranges: [{ start: 1, end: 10 }], bytes: 100 }],
|
||||
sourceBytes: 100,
|
||||
responseBytes: 400,
|
||||
...over,
|
||||
};
|
||||
}
|
||||
|
||||
describe('ExploreSessionState — the container', () => {
|
||||
it('counts calls per project and hands back a 1-based session index', () => {
|
||||
const state = new ExploreSessionState();
|
||||
expect(state.record(emission('/repo/a'))?.index).toBe(1);
|
||||
expect(state.record(emission('/repo/a'))?.index).toBe(2);
|
||||
expect(state.callCount('/repo/a')).toBe(2);
|
||||
expect(state.forProject('/repo/a')?.responseBytes).toBe(800);
|
||||
});
|
||||
|
||||
it('keys state per project — a second project starts its own count', () => {
|
||||
const state = new ExploreSessionState();
|
||||
state.record(emission('/repo/a'));
|
||||
state.record(emission('/repo/a'));
|
||||
expect(state.record(emission('/repo/b'))?.index).toBe(1);
|
||||
expect(state.callCount('/repo/a')).toBe(2);
|
||||
expect(state.callCount('/repo/b')).toBe(1);
|
||||
expect(state.forProject('/repo/b')?.calls).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('treats trailing slashes and `.` segments as the same project', () => {
|
||||
const state = new ExploreSessionState();
|
||||
state.record(emission('/repo/a'));
|
||||
state.record(emission('/repo/a/'));
|
||||
state.record(emission('/repo/a/./'));
|
||||
expect(state.callCount('/repo/a')).toBe(3);
|
||||
expect(state.snapshot()).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('never reports a project it was never told about', () => {
|
||||
const state = new ExploreSessionState();
|
||||
expect(state.forProject('/never/queried')).toBeNull();
|
||||
expect(state.callCount('/never/queried')).toBe(0);
|
||||
});
|
||||
|
||||
it('keeps counting past the retained-call bound — decay must not reset itself', () => {
|
||||
const state = new ExploreSessionState();
|
||||
const total = EXPLORE_SESSION_LIMITS.MAX_CALLS_RETAINED + 5;
|
||||
for (let i = 0; i < total; i++) state.record(emission('/repo/a'));
|
||||
const project = state.forProject('/repo/a')!;
|
||||
expect(project.callCount).toBe(total);
|
||||
expect(project.calls).toHaveLength(EXPLORE_SESSION_LIMITS.MAX_CALLS_RETAINED);
|
||||
// Detail is dropped from the OLDEST end; the newest call is always retained.
|
||||
expect(project.calls[project.calls.length - 1]!.index).toBe(total);
|
||||
expect(project.calls[0]!.index).toBe(total - EXPLORE_SESSION_LIMITS.MAX_CALLS_RETAINED + 1);
|
||||
});
|
||||
|
||||
it('bounds the number of projects, evicting the least recently used', () => {
|
||||
const state = new ExploreSessionState();
|
||||
const roots = Array.from({ length: EXPLORE_SESSION_LIMITS.MAX_PROJECTS + 2 }, (_, i) => `/repo/${i}`);
|
||||
for (const root of roots) state.record(emission(root));
|
||||
expect(state.snapshot()).toHaveLength(EXPLORE_SESSION_LIMITS.MAX_PROJECTS);
|
||||
expect(state.forProject(roots[0]!)).toBeNull();
|
||||
expect(state.forProject(roots[roots.length - 1]!)).not.toBeNull();
|
||||
});
|
||||
|
||||
it('keeps a re-queried project alive past newer ones', () => {
|
||||
const state = new ExploreSessionState();
|
||||
const roots = Array.from({ length: EXPLORE_SESSION_LIMITS.MAX_PROJECTS }, (_, i) => `/repo/${i}`);
|
||||
for (const root of roots) state.record(emission(root));
|
||||
state.record(emission(roots[0]!)); // touch the oldest
|
||||
state.record(emission('/repo/newcomer')); // forces one eviction
|
||||
expect(state.forProject(roots[0]!)?.callCount).toBe(2);
|
||||
expect(state.forProject(roots[1]!)).toBeNull();
|
||||
});
|
||||
|
||||
it('bounds files per call, keeping the ones that got the most source', () => {
|
||||
const state = new ExploreSessionState();
|
||||
const files = Array.from({ length: EXPLORE_SESSION_LIMITS.MAX_FILES_PER_CALL + 6 }, (_, i) => ({
|
||||
path: `f${i}.ts`,
|
||||
ranges: [{ start: 1, end: 5 }],
|
||||
bytes: i + 1,
|
||||
}));
|
||||
state.record(emission('/repo/a', { files }));
|
||||
const kept = state.forProject('/repo/a')!.calls[0]!.files;
|
||||
expect(kept).toHaveLength(EXPLORE_SESSION_LIMITS.MAX_FILES_PER_CALL);
|
||||
expect(kept.map((f) => f.path)).toContain(`f${files.length - 1}.ts`);
|
||||
expect(kept.map((f) => f.path)).not.toContain('f0.ts');
|
||||
});
|
||||
|
||||
it('ignores an emission with no project root rather than filing it under ""', () => {
|
||||
const state = new ExploreSessionState();
|
||||
expect(state.record({ ...emission(''), projectRoot: '' })).toBeNull();
|
||||
expect(state.snapshot()).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('hands out copies — a caller cannot mutate the record it read', () => {
|
||||
const state = new ExploreSessionState();
|
||||
state.record(emission('/repo/a'));
|
||||
const snap = state.forProject('/repo/a')!;
|
||||
snap.calls[0]!.files[0]!.ranges.push({ start: 999, end: 1000 });
|
||||
expect(state.forProject('/repo/a')!.calls[0]!.files[0]!.ranges).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('view() carries only the most recent calls per project', () => {
|
||||
const state = new ExploreSessionState();
|
||||
for (let i = 0; i < EXPLORE_SESSION_LIMITS.MAX_CALLS_RETAINED; i++) state.record(emission('/repo/a'));
|
||||
const view = state.view();
|
||||
expect(view.projects[0]!.callCount).toBe(EXPLORE_SESSION_LIMITS.MAX_CALLS_RETAINED);
|
||||
expect(view.projects[0]!.calls).toHaveLength(EXPLORE_SESSION_LIMITS.MAX_VIEW_CALLS);
|
||||
expect(viewForProject(view, '/repo/a')?.callCount).toBe(EXPLORE_SESSION_LIMITS.MAX_CALLS_RETAINED);
|
||||
// A tracked session that hasn't touched this project yet reads as EMPTY,
|
||||
// not untracked — only a missing view (nobody tracking) is null.
|
||||
expect(viewForProject(view, '/repo/other')?.callCount).toBe(0);
|
||||
expect(viewForProject(null, '/repo/a')).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('range bookkeeping', () => {
|
||||
it('merges overlapping and adjacent spans into one', () => {
|
||||
const { ranges, truncated } = coalesceRanges([
|
||||
{ start: 10, end: 20 },
|
||||
{ start: 15, end: 25 }, // overlaps
|
||||
{ start: 26, end: 30 }, // adjacent — one contiguous block of source
|
||||
{ start: 60, end: 61 },
|
||||
]);
|
||||
expect(ranges).toEqual([{ start: 10, end: 30 }, { start: 60, end: 61 }]);
|
||||
expect(truncated).toBe(false);
|
||||
});
|
||||
|
||||
it('drops junk spans instead of recording a range that was never served', () => {
|
||||
const { ranges } = coalesceRanges([
|
||||
{ start: 5, end: 1 }, // inverted
|
||||
{ start: 0, end: 3 }, // before line 1
|
||||
{ start: NaN, end: 4 },
|
||||
{ start: 7, end: 9 },
|
||||
]);
|
||||
expect(ranges).toEqual([{ start: 7, end: 9 }]);
|
||||
});
|
||||
|
||||
it('caps the range list by KEEPING the largest spans, and says it truncated', () => {
|
||||
// Spaced far enough apart that none of them merge — this is about the cap.
|
||||
const many = Array.from({ length: EXPLORE_SESSION_LIMITS.MAX_RANGES_PER_FILE + 5 }, (_, i) => ({
|
||||
start: i * 200 + 1,
|
||||
end: i * 200 + 2 + i, // later spans are longer
|
||||
}));
|
||||
const { ranges, truncated } = coalesceRanges(many);
|
||||
expect(truncated).toBe(true);
|
||||
expect(ranges).toHaveLength(EXPLORE_SESSION_LIMITS.MAX_RANGES_PER_FILE);
|
||||
// Still in line order, and the biggest span survived.
|
||||
expect(ranges.map((r) => r.start)).toEqual([...ranges.map((r) => r.start)].sort((a, b) => a - b));
|
||||
expect(ranges.some((r) => r.start === many[many.length - 1]!.start)).toBe(true);
|
||||
});
|
||||
|
||||
it('flags truncation on the stored record so a consumer knows it under-knows', () => {
|
||||
const state = new ExploreSessionState();
|
||||
const ranges = Array.from({ length: EXPLORE_SESSION_LIMITS.MAX_RANGES_PER_FILE + 3 }, (_, i) => ({
|
||||
start: i * 10 + 1, end: i * 10 + 4,
|
||||
}));
|
||||
state.record(emission('/repo/a', { files: [{ path: 'big.ts', ranges, bytes: 900 }] }));
|
||||
expect(state.forProject('/repo/a')!.calls[0]!.files[0]!.rangesTruncated).toBe(true);
|
||||
});
|
||||
|
||||
it('answers whether a line was already served', () => {
|
||||
const ranges = [{ start: 10, end: 20 }, { start: 40, end: 41 }];
|
||||
expect(rangesCover(ranges, 10)).toBe(true);
|
||||
expect(rangesCover(ranges, 20)).toBe(true);
|
||||
expect(rangesCover(ranges, 21)).toBe(false);
|
||||
expect(rangesCover(ranges, 40)).toBe(true);
|
||||
});
|
||||
|
||||
it('folds case only on the case-insensitive platforms', () => {
|
||||
const insensitive = process.platform === 'darwin' || process.platform === 'win32';
|
||||
expect(exploreProjectKey('/Repo/A') === exploreProjectKey('/repo/a')).toBe(insensitive);
|
||||
});
|
||||
});
|
||||
|
||||
describe('session view arriving on tool args', () => {
|
||||
it('reads a well-formed view and ignores anything else', () => {
|
||||
const state = new ExploreSessionState();
|
||||
state.record(emission('/repo/a'));
|
||||
expect(readExploreSessionView({ [EXPLORE_SESSION_VIEW_ARG]: state.view() })?.projects).toHaveLength(1);
|
||||
expect(readExploreSessionView({})).toBeNull();
|
||||
expect(readExploreSessionView({ [EXPLORE_SESSION_VIEW_ARG]: 'nope' })).toBeNull();
|
||||
expect(readExploreSessionView({ [EXPLORE_SESSION_VIEW_ARG]: { projects: 'nope' } })).toBeNull();
|
||||
});
|
||||
|
||||
it('drops malformed project entries rather than trusting them', () => {
|
||||
const view = readExploreSessionView({
|
||||
[EXPLORE_SESSION_VIEW_ARG]: { projects: [{ projectRoot: '/repo/a', calls: [] }, { nope: 1 }, null] },
|
||||
});
|
||||
expect(view?.projects).toHaveLength(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe('explore records what it actually served', () => {
|
||||
let testDir: string;
|
||||
let cg: CodeGraph;
|
||||
let handler: ToolHandler;
|
||||
|
||||
beforeAll(async () => {
|
||||
testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cg17-'));
|
||||
fs.cpSync(FIXTURE_SRC, testDir, { recursive: true });
|
||||
fs.rmSync(path.join(testDir, '.codegraph'), { recursive: true, force: true });
|
||||
cg = CodeGraph.initSync(testDir);
|
||||
await cg.indexAll();
|
||||
handler = new ToolHandler(cg);
|
||||
}, 120_000);
|
||||
|
||||
afterAll(() => {
|
||||
if (cg) cg.destroy();
|
||||
if (testDir && fs.existsSync(testDir)) fs.rmSync(testDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('files one record per call, with the files and line ranges it emitted', async () => {
|
||||
const session = new ExploreSessionState();
|
||||
await handler.execute('codegraph_explore', { query: QUERY }, session);
|
||||
|
||||
const project = session.forProject(cg.getProjectRoot());
|
||||
expect(project).not.toBeNull();
|
||||
expect(project!.callCount).toBe(1);
|
||||
|
||||
const call = project!.calls[0]!;
|
||||
expect(call.files.length).toBeGreaterThan(0);
|
||||
expect(call.sourceBytes).toBeGreaterThan(0);
|
||||
expect(call.responseBytes).toBeGreaterThan(call.sourceBytes);
|
||||
for (const file of call.files) {
|
||||
expect(file.ranges.length).toBeGreaterThan(0);
|
||||
for (const r of file.ranges) {
|
||||
expect(r.start).toBeGreaterThanOrEqual(1);
|
||||
expect(r.end).toBeGreaterThanOrEqual(r.start);
|
||||
}
|
||||
}
|
||||
}, 60_000);
|
||||
|
||||
it('records only files whose source is really in the response', async () => {
|
||||
const session = new ExploreSessionState();
|
||||
const result = await handler.execute('codegraph_explore', { query: QUERY }, session);
|
||||
const text = result.content[0]!.text;
|
||||
for (const file of session.forProject(cg.getProjectRoot())!.calls[0]!.files) {
|
||||
expect(text).toContain(file.path);
|
||||
}
|
||||
}, 60_000);
|
||||
|
||||
it('the recorded ranges name lines that are really in the emitted source', async () => {
|
||||
const session = new ExploreSessionState();
|
||||
await handler.execute('codegraph_explore', { query: QUERY }, session);
|
||||
for (const file of session.forProject(cg.getProjectRoot())!.calls[0]!.files) {
|
||||
const lineCount = fs.readFileSync(path.join(testDir, file.path), 'utf-8').split('\n').length;
|
||||
for (const r of file.ranges) expect(r.end).toBeLessThanOrEqual(lineCount);
|
||||
}
|
||||
}, 60_000);
|
||||
|
||||
it('leaves the agent-facing response untouched — no side-channel on the wire', async () => {
|
||||
const session = new ExploreSessionState();
|
||||
const tracked = await handler.execute('codegraph_explore', { query: QUERY }, session);
|
||||
const untracked = await handler.execute('codegraph_explore', { query: QUERY });
|
||||
|
||||
expect(tracked.content[0]!.text).toBe(untracked.content[0]!.text);
|
||||
for (const result of [tracked, untracked]) {
|
||||
expect(EXPLORE_EMISSION_KEY in result).toBe(false);
|
||||
expect(JSON.stringify(result)).not.toContain(EXPLORE_EMISSION_KEY);
|
||||
}
|
||||
}, 60_000);
|
||||
|
||||
it('ignores a session view a client spelled itself — the record is the server\'s', async () => {
|
||||
const forged = {
|
||||
projects: [{ projectRoot: cg.getProjectRoot(), callCount: 99, responseBytes: 1e6, calls: [] }],
|
||||
};
|
||||
const result = await handler.execute('codegraph_explore', {
|
||||
query: QUERY,
|
||||
[EXPLORE_SESSION_VIEW_ARG]: forged,
|
||||
});
|
||||
const clean = await handler.execute('codegraph_explore', { query: QUERY });
|
||||
expect(result.content[0]!.text).toBe(clean.content[0]!.text);
|
||||
}, 60_000);
|
||||
|
||||
it('counts an empty answer as a call, since it still spends the tier budget', async () => {
|
||||
const session = new ExploreSessionState();
|
||||
await handler.execute('codegraph_explore', { query: 'zzqqxx_no_such_symbol_anywhere' }, session);
|
||||
const project = session.forProject(cg.getProjectRoot());
|
||||
expect(project?.callCount).toBe(1);
|
||||
expect(project?.calls[0]!.files).toHaveLength(0);
|
||||
}, 60_000);
|
||||
|
||||
it('two sessions on ONE handler never see each other\'s calls', async () => {
|
||||
const a = new ExploreSessionState();
|
||||
const b = new ExploreSessionState();
|
||||
await handler.execute('codegraph_explore', { query: QUERY }, a);
|
||||
await handler.execute('codegraph_explore', { query: QUERY }, a);
|
||||
await handler.execute('codegraph_explore', { query: QUERY }, b);
|
||||
|
||||
expect(a.callCount(cg.getProjectRoot())).toBe(2);
|
||||
expect(b.callCount(cg.getProjectRoot())).toBe(1);
|
||||
}, 90_000);
|
||||
|
||||
it('a caller that tracks nothing still gets a clean result', async () => {
|
||||
const result = await handler.execute('codegraph_explore', { query: QUERY });
|
||||
expect(result.isError).toBeFalsy();
|
||||
expect(result.content[0]!.text.length).toBeGreaterThan(0);
|
||||
}, 60_000);
|
||||
|
||||
it('reports the session state through the CG-4 diagnostic', async () => {
|
||||
const sidecar = path.join(testDir, 'cg17-diagnostic.jsonl');
|
||||
const session = new ExploreSessionState();
|
||||
const previous = process.env.CODEGRAPH_EXPLORE_DEBUG;
|
||||
process.env.CODEGRAPH_EXPLORE_DEBUG = sidecar;
|
||||
try {
|
||||
await handler.execute('codegraph_explore', { query: QUERY }, session);
|
||||
await handler.execute('codegraph_explore', { query: QUERY }, session);
|
||||
} finally {
|
||||
if (previous === undefined) delete process.env.CODEGRAPH_EXPLORE_DEBUG;
|
||||
else process.env.CODEGRAPH_EXPLORE_DEBUG = previous;
|
||||
}
|
||||
|
||||
const reports = fs.readFileSync(sidecar, 'utf-8').trim().split('\n').map((l) => JSON.parse(l));
|
||||
expect(reports).toHaveLength(2);
|
||||
// The first call is the session's first: nothing served before it.
|
||||
expect(reports[0].session).toEqual({
|
||||
callIndex: 1, priorCalls: 0, priorResponseChars: 0, priorFiles: [],
|
||||
});
|
||||
// The second sees the first call's files and their ranges.
|
||||
expect(reports[1].session.callIndex).toBe(2);
|
||||
expect(reports[1].session.priorCalls).toBe(1);
|
||||
expect(reports[1].session.priorResponseChars).toBeGreaterThan(0);
|
||||
expect(reports[1].session.priorFiles.length).toBeGreaterThan(0);
|
||||
expect(reports[1].session.priorFiles[0].ranges[0]).toHaveLength(2);
|
||||
}, 90_000);
|
||||
|
||||
it('omits the session block entirely when the caller tracks no state', async () => {
|
||||
const sidecar = path.join(testDir, 'cg17-untracked.jsonl');
|
||||
const previous = process.env.CODEGRAPH_EXPLORE_DEBUG;
|
||||
process.env.CODEGRAPH_EXPLORE_DEBUG = sidecar;
|
||||
try {
|
||||
await handler.execute('codegraph_explore', { query: QUERY });
|
||||
} finally {
|
||||
if (previous === undefined) delete process.env.CODEGRAPH_EXPLORE_DEBUG;
|
||||
else process.env.CODEGRAPH_EXPLORE_DEBUG = previous;
|
||||
}
|
||||
const report = JSON.parse(fs.readFileSync(sidecar, 'utf-8').trim());
|
||||
expect(report.session).toBeUndefined();
|
||||
}, 60_000);
|
||||
|
||||
it('keys on the RESOLVED root, not the path the agent typed', async () => {
|
||||
// The same project reached two ways — bare, and via a `projectPath` pointing
|
||||
// at a subdirectory. Both resolve to one index, so both must land in one
|
||||
// bucket; keying on the typed path would split a session's history in two
|
||||
// and hand a later call a half-empty record.
|
||||
//
|
||||
// (Two genuinely DIFFERENT projects can't be exercised here: opening a
|
||||
// second index inside vitest fails on the lazy `require('../index')` — see
|
||||
// the ToolHandler cache notes. The container-level tests above cover the
|
||||
// multi-project keying itself.)
|
||||
const session = new ExploreSessionState();
|
||||
await handler.execute('codegraph_explore', { query: QUERY }, session);
|
||||
await handler.execute(
|
||||
'codegraph_explore',
|
||||
{ query: QUERY, projectPath: path.join(testDir, 'internal') },
|
||||
session,
|
||||
);
|
||||
|
||||
expect(session.snapshot()).toHaveLength(1);
|
||||
expect(session.callCount(cg.getProjectRoot())).toBe(2);
|
||||
}, 90_000);
|
||||
});
|
||||
|
||||
describe('sessions sharing a daemon', () => {
|
||||
/** Minimal transport: captures the message handler so a test can drive it. */
|
||||
function fakeTransport(): JsonRpcTransport & { deliver: (m: JsonRpcRequest) => Promise<void>; results: unknown[] } {
|
||||
let handle: ((m: JsonRpcRequest | JsonRpcNotification) => Promise<void>) | null = null;
|
||||
const results: unknown[] = [];
|
||||
return {
|
||||
start(h) { handle = h as typeof handle; },
|
||||
stop() { /* nothing to tear down */ },
|
||||
send() { /* unused */ },
|
||||
notify() { /* unused */ },
|
||||
async request() { return {}; },
|
||||
sendResult(_id, result) { results.push(result); },
|
||||
sendError() { /* unused */ },
|
||||
results,
|
||||
async deliver(m: JsonRpcRequest) { await handle?.(m); },
|
||||
};
|
||||
}
|
||||
|
||||
it('give each session its own state, and one session\'s calls stay there', async () => {
|
||||
const calls: Array<ExploreSessionState | undefined> = [];
|
||||
// A ToolHandler stand-in: the point here is WHICH state object arrives, not
|
||||
// what explore returns, so a real index would only slow the assertion down.
|
||||
const handler = {
|
||||
getTools: () => [],
|
||||
execute: async (_tool: string, _args: Record<string, unknown>, state?: ExploreSessionState) => {
|
||||
calls.push(state);
|
||||
state?.record(emission('/repo/shared'));
|
||||
return { content: [{ type: 'text' as const, text: 'ok' }] };
|
||||
},
|
||||
};
|
||||
const engine = {
|
||||
ensureInitialized: async () => { /* already open */ },
|
||||
hasDefaultCodeGraph: () => true,
|
||||
getProjectPath: () => '/repo/shared',
|
||||
retryInitializeSync: () => { /* nothing to retry */ },
|
||||
getToolHandler: () => handler,
|
||||
} as unknown as MCPEngine;
|
||||
|
||||
const transportA = fakeTransport();
|
||||
const transportB = fakeTransport();
|
||||
const sessionA = new MCPSession(transportA, engine);
|
||||
const sessionB = new MCPSession(transportB, engine);
|
||||
sessionA.start();
|
||||
sessionB.start();
|
||||
|
||||
expect(sessionA.getExploreSessionState()).not.toBe(sessionB.getExploreSessionState());
|
||||
|
||||
const call = (id: number): JsonRpcRequest => ({
|
||||
jsonrpc: '2.0', id, method: 'tools/call',
|
||||
params: { name: 'codegraph_explore', arguments: { query: 'q' } },
|
||||
});
|
||||
await transportA.deliver(call(1));
|
||||
await transportA.deliver(call(2));
|
||||
await transportB.deliver(call(3));
|
||||
|
||||
expect(calls[0]).toBe(sessionA.getExploreSessionState());
|
||||
expect(calls[2]).toBe(sessionB.getExploreSessionState());
|
||||
expect(sessionA.getExploreSessionState().callCount('/repo/shared')).toBe(2);
|
||||
expect(sessionB.getExploreSessionState().callCount('/repo/shared')).toBe(1);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"name": "ambient-decls-ts-fixture",
|
||||
"private": true,
|
||||
"version": "0.0.0",
|
||||
"type": "module",
|
||||
"description": "CG-28 fixture — declaration-only files competing with implementation for one explore envelope."
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
export interface BucketObject {
|
||||
key: string;
|
||||
body: ReadableStream<Uint8Array>;
|
||||
size: number;
|
||||
}
|
||||
|
||||
export interface Bucket {
|
||||
put(
|
||||
key: string,
|
||||
value: ReadableStream<Uint8Array>,
|
||||
options?: { httpMetadata?: { contentType?: string } },
|
||||
): Promise<void>;
|
||||
get(key: string): Promise<BucketObject | null>;
|
||||
}
|
||||
|
||||
export interface MetadataStore {
|
||||
put(id: string, value: string): Promise<void>;
|
||||
get(id: string): Promise<string | null>;
|
||||
}
|
||||
|
||||
const objects = new Map<string, BucketObject>();
|
||||
const rows = new Map<string, string>();
|
||||
|
||||
/** The object-storage binding. */
|
||||
export function openBucket(): Bucket {
|
||||
return {
|
||||
async put(key, value) {
|
||||
objects.set(key, { key, body: value, size: 0 });
|
||||
},
|
||||
async get(key) {
|
||||
return objects.get(key) ?? null;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/** The metadata key-value binding. */
|
||||
export function openMetadataStore(): MetadataStore {
|
||||
return {
|
||||
async put(id, value) {
|
||||
rows.set(id, value);
|
||||
},
|
||||
async get(id) {
|
||||
return rows.get(id) ?? null;
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
export interface UploadMessageBody {
|
||||
key: string;
|
||||
metadataId: string;
|
||||
contentType: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Publish the follow-up message for a stored upload. Batched so a burst of
|
||||
* uploads does not open one producer call per object.
|
||||
*/
|
||||
export async function enqueueUploadMessage(body: UploadMessageBody): Promise<void> {
|
||||
const queue = openUploadQueue();
|
||||
await queue.send(body, { contentType: 'json' });
|
||||
}
|
||||
|
||||
/** Consumer side: process a batch of upload messages. */
|
||||
export async function consumeUploadBatch(messages: UploadMessageBody[]): Promise<number> {
|
||||
let handled = 0;
|
||||
for (const message of messages) {
|
||||
if (!message.key) continue;
|
||||
handled += 1;
|
||||
}
|
||||
return handled;
|
||||
}
|
||||
|
||||
interface UploadQueue {
|
||||
send(body: UploadMessageBody, options: { contentType: string }): Promise<void>;
|
||||
}
|
||||
|
||||
/** The binding lookup, isolated so tests can swap it. */
|
||||
export function openUploadQueue(): UploadQueue {
|
||||
return {
|
||||
async send() {
|
||||
/* binding provided by the runtime */
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,44 @@
|
||||
export interface ParsedUpload {
|
||||
ok: true;
|
||||
key: string;
|
||||
body: ReadableStream<Uint8Array>;
|
||||
contentType: string;
|
||||
width: number;
|
||||
height: number;
|
||||
format: string;
|
||||
}
|
||||
|
||||
export interface ParseFailure {
|
||||
ok: false;
|
||||
error: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Pull the object key, declared dimensions and the raw body stream off an
|
||||
* upload request. Never buffers the body — the stream is handed straight to
|
||||
* the storage layer.
|
||||
*/
|
||||
export async function parseUploadRequest(
|
||||
request: Request,
|
||||
): Promise<ParsedUpload | ParseFailure> {
|
||||
const url = new URL(request.url);
|
||||
const key = url.searchParams.get('key');
|
||||
if (!key) return { ok: false, error: 'missing key' };
|
||||
if (!request.body) return { ok: false, error: 'missing body' };
|
||||
|
||||
return {
|
||||
ok: true,
|
||||
key,
|
||||
body: request.body as ReadableStream<Uint8Array>,
|
||||
contentType: request.headers.get('content-type') ?? 'application/octet-stream',
|
||||
width: numberParam(url, 'width'),
|
||||
height: numberParam(url, 'height'),
|
||||
format: url.searchParams.get('format') ?? 'jpeg',
|
||||
};
|
||||
}
|
||||
|
||||
function numberParam(url: URL, name: string): number {
|
||||
const raw = url.searchParams.get(name);
|
||||
const parsed = raw ? Number.parseInt(raw, 10) : 0;
|
||||
return Number.isFinite(parsed) ? parsed : 0;
|
||||
}
|
||||
@@ -0,0 +1,66 @@
|
||||
import { streamBodyToStorage } from '../storage/stream.js';
|
||||
import { recordImageMetadata } from '../storage/metadata.js';
|
||||
import { enqueueUploadMessage } from '../lib/queue.js';
|
||||
import { parseUploadRequest } from '../lib/request.js';
|
||||
|
||||
export interface UploadResult {
|
||||
key: string;
|
||||
bytes: number;
|
||||
contentType: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Entry point for an upload request: parse it, stream the body into object
|
||||
* storage, record the image metadata, then queue the follow-up work.
|
||||
*/
|
||||
export async function handleUploadRequest(request: Request): Promise<Response> {
|
||||
const parsed = await parseUploadRequest(request);
|
||||
if (!parsed.ok) {
|
||||
return new Response(JSON.stringify({ error: parsed.error }), { status: 400 });
|
||||
}
|
||||
|
||||
const stored = await streamBodyToStorage(parsed.body, parsed.key, parsed.contentType);
|
||||
const metadata = await recordImageMetadata(stored.key, {
|
||||
width: parsed.width,
|
||||
height: parsed.height,
|
||||
format: parsed.format,
|
||||
bytes: stored.bytes,
|
||||
});
|
||||
|
||||
await enqueueUploadMessage({
|
||||
key: stored.key,
|
||||
metadataId: metadata.id,
|
||||
contentType: stored.contentType,
|
||||
});
|
||||
|
||||
return new Response(JSON.stringify(summarizeUpload(stored, metadata.id)), {
|
||||
status: 201,
|
||||
headers: { 'content-type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
/** Shape the client sees back after a successful upload. */
|
||||
export function summarizeUpload(stored: UploadResult, metadataId: string) {
|
||||
return {
|
||||
key: stored.key,
|
||||
bytes: stored.bytes,
|
||||
contentType: stored.contentType,
|
||||
metadataId,
|
||||
};
|
||||
}
|
||||
|
||||
/** Reject uploads whose declared size exceeds the per-account ceiling. */
|
||||
export function isWithinUploadLimit(bytes: number, limit: number): boolean {
|
||||
if (!Number.isFinite(bytes) || bytes < 0) return false;
|
||||
return bytes <= limit;
|
||||
}
|
||||
|
||||
/** Delete-side counterpart, kept here so the route module is not a one-liner. */
|
||||
export async function handleDeleteRequest(request: Request, key: string): Promise<Response> {
|
||||
const parsed = await parseUploadRequest(request);
|
||||
if (!parsed.ok) {
|
||||
return new Response(JSON.stringify({ error: parsed.error }), { status: 400 });
|
||||
}
|
||||
await enqueueUploadMessage({ key, metadataId: '', contentType: 'application/x-delete' });
|
||||
return new Response(null, { status: 204 });
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
import { openMetadataStore } from '../lib/bucket.js';
|
||||
|
||||
export interface ImageMetadataInput {
|
||||
width: number;
|
||||
height: number;
|
||||
format: string;
|
||||
bytes: number;
|
||||
}
|
||||
|
||||
export interface ImageMetadataRecord extends ImageMetadataInput {
|
||||
id: string;
|
||||
key: string;
|
||||
recordedAt: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Record the image metadata for a stored object. Writes go to the metadata
|
||||
* store keyed by object key; the returned record carries the id the queue
|
||||
* message references.
|
||||
*/
|
||||
export async function recordImageMetadata(
|
||||
key: string,
|
||||
input: ImageMetadataInput,
|
||||
): Promise<ImageMetadataRecord> {
|
||||
const store = openMetadataStore();
|
||||
const record: ImageMetadataRecord = {
|
||||
...input,
|
||||
id: metadataIdFor(key, input),
|
||||
key,
|
||||
recordedAt: 0,
|
||||
};
|
||||
await store.put(record.id, JSON.stringify(record));
|
||||
return record;
|
||||
}
|
||||
|
||||
/** Deterministic id so a retried upload records the same metadata row. */
|
||||
export function metadataIdFor(key: string, input: ImageMetadataInput): string {
|
||||
return `${key}:${input.format}:${input.width}x${input.height}`;
|
||||
}
|
||||
|
||||
/** Read a metadata record back for the download and listing paths. */
|
||||
export async function loadImageMetadata(id: string): Promise<ImageMetadataRecord | null> {
|
||||
const store = openMetadataStore();
|
||||
const raw = await store.get(id);
|
||||
return raw ? (JSON.parse(raw) as ImageMetadataRecord) : null;
|
||||
}
|
||||
|
||||
/** Normalize a client-declared format string to the canonical set. */
|
||||
export function normalizeFormat(format: string): string {
|
||||
const lowered = format.trim().toLowerCase();
|
||||
if (lowered === 'jpg') return 'jpeg';
|
||||
if (lowered === 'tif') return 'tiff';
|
||||
return lowered;
|
||||
}
|
||||
@@ -0,0 +1,86 @@
|
||||
import { openBucket } from '../lib/bucket.js';
|
||||
import type { StorageFailure, UploadTelemetry } from './types.js';
|
||||
|
||||
export interface StoredObject {
|
||||
key: string;
|
||||
bytes: number;
|
||||
contentType: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Stream a request body into object storage without buffering it in memory.
|
||||
* The body is piped through a counting transform so the byte total is known
|
||||
* by the time the put resolves.
|
||||
*/
|
||||
export async function streamBodyToStorage(
|
||||
body: ReadableStream<Uint8Array>,
|
||||
key: string,
|
||||
contentType: string,
|
||||
): Promise<StoredObject> {
|
||||
const bucket = openBucket();
|
||||
const counter = createByteCounter();
|
||||
const piped = body.pipeThrough(counter.transform, { preventClose: false });
|
||||
|
||||
await bucket.put(key, piped, { httpMetadata: { contentType } });
|
||||
|
||||
return { key, bytes: counter.total(), contentType };
|
||||
}
|
||||
|
||||
/**
|
||||
* A transform stream that counts the bytes flowing through it. Separated from
|
||||
* the pipe above so the byte total can be read after the stream settles.
|
||||
*/
|
||||
export function createByteCounter() {
|
||||
let total = 0;
|
||||
const transform = new TransformStream<Uint8Array, Uint8Array>({
|
||||
transform(chunk, controller) {
|
||||
total += chunk.byteLength;
|
||||
controller.enqueue(chunk);
|
||||
},
|
||||
});
|
||||
return { transform, total: () => total };
|
||||
}
|
||||
|
||||
/**
|
||||
* Read a stored object back out of the bucket as a stream, for the download
|
||||
* path. Mirrors the upload side so both directions live in one module.
|
||||
*/
|
||||
export async function readObjectStream(key: string): Promise<ReadableStream<Uint8Array> | null> {
|
||||
const bucket = openBucket();
|
||||
const object = await bucket.get(key);
|
||||
if (!object) return null;
|
||||
return object.body;
|
||||
}
|
||||
|
||||
/** Timing/retry record for one stored object, handed to the metrics sink. */
|
||||
export function telemetryFor(stored: StoredObject, durationMs: number): UploadTelemetry {
|
||||
return { key: stored.key, bytes: stored.bytes, durationMs, retries: 0 };
|
||||
}
|
||||
|
||||
/** Describe a failed stage so the caller can report it without re-deriving it. */
|
||||
export function storageFailure(
|
||||
key: string,
|
||||
stage: StorageFailure['stage'],
|
||||
message: string,
|
||||
): StorageFailure {
|
||||
return { key, stage, message };
|
||||
}
|
||||
|
||||
/** Cap a stream at `limit` bytes, erroring out rather than storing an overrun. */
|
||||
export function limitStream(
|
||||
source: ReadableStream<Uint8Array>,
|
||||
limit: number,
|
||||
): ReadableStream<Uint8Array> {
|
||||
let seen = 0;
|
||||
const guard = new TransformStream<Uint8Array, Uint8Array>({
|
||||
transform(chunk, controller) {
|
||||
seen += chunk.byteLength;
|
||||
if (seen > limit) {
|
||||
controller.error(new Error(`upload exceeded ${limit} bytes`));
|
||||
return;
|
||||
}
|
||||
controller.enqueue(chunk);
|
||||
},
|
||||
});
|
||||
return source.pipeThrough(guard);
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
/**
|
||||
* Shared shapes for the storage layer. Declaration-only like the ambient files
|
||||
* under `types/` — but the modules that answer a flow question are typed BY it,
|
||||
* so it is part of that answer's structure rather than a global shim.
|
||||
*/
|
||||
|
||||
export interface UploadTelemetry {
|
||||
key: string;
|
||||
bytes: number;
|
||||
durationMs: number;
|
||||
retries: number;
|
||||
}
|
||||
|
||||
export interface StorageFailure {
|
||||
key: string;
|
||||
stage: 'parse' | 'stream' | 'metadata' | 'queue';
|
||||
message: string;
|
||||
}
|
||||
@@ -0,0 +1,212 @@
|
||||
// Hand-maintained ambient declarations for the parts of the platform our
|
||||
// runtime exposes but the published typings do not cover yet. Edit freely —
|
||||
// nothing regenerates this file. Kept alongside the app so module augmentation
|
||||
// and the global shims live in one place.
|
||||
|
||||
declare global {
|
||||
interface UploadStorage {
|
||||
put(
|
||||
key: string,
|
||||
body: ReadableStream<Uint8Array>,
|
||||
options?: UploadPutOptions,
|
||||
): Promise<StoredUploadObject>;
|
||||
get(key: string): Promise<StoredUploadObject | null>;
|
||||
head(key: string): Promise<StoredUploadHead | null>;
|
||||
delete(key: string | string[]): Promise<void>;
|
||||
list(options?: UploadListOptions): Promise<UploadListResult>;
|
||||
}
|
||||
|
||||
interface StoredUploadObject {
|
||||
readonly key: string;
|
||||
readonly size: number;
|
||||
readonly etag: string;
|
||||
readonly uploaded: Date;
|
||||
readonly body: ReadableStream<Uint8Array>;
|
||||
readonly contentType: string;
|
||||
readonly metadata?: ImageMetadataShim;
|
||||
arrayBuffer(): Promise<ArrayBuffer>;
|
||||
text(): Promise<string>;
|
||||
json<T>(): Promise<T>;
|
||||
}
|
||||
|
||||
interface StoredUploadHead {
|
||||
readonly key: string;
|
||||
readonly size: number;
|
||||
readonly etag: string;
|
||||
readonly uploaded: Date;
|
||||
readonly contentType: string;
|
||||
}
|
||||
|
||||
interface UploadPutOptions {
|
||||
contentType?: string;
|
||||
cacheControl?: string;
|
||||
customMetadata?: Record<string, string>;
|
||||
checksum?: string;
|
||||
storageClass?: 'standard' | 'infrequent';
|
||||
}
|
||||
|
||||
interface UploadListOptions {
|
||||
prefix?: string;
|
||||
cursor?: string;
|
||||
limit?: number;
|
||||
delimiter?: string;
|
||||
include?: ('metadata' | 'contentType')[];
|
||||
}
|
||||
|
||||
interface UploadListResult {
|
||||
objects: StoredUploadHead[];
|
||||
truncated: boolean;
|
||||
cursor?: string;
|
||||
prefixes: string[];
|
||||
}
|
||||
|
||||
interface ImageMetadataShim {
|
||||
format: string;
|
||||
fileSize: number;
|
||||
width: number;
|
||||
height: number;
|
||||
orientation?: number;
|
||||
colorSpace?: string;
|
||||
}
|
||||
|
||||
interface MetadataRowShim {
|
||||
id: string;
|
||||
key: string;
|
||||
recordedAt: number;
|
||||
format: string;
|
||||
bytes: number;
|
||||
width: number;
|
||||
height: number;
|
||||
}
|
||||
|
||||
interface MetadataStoreShim {
|
||||
put(id: string, value: string, options?: MetadataPutOptions): Promise<void>;
|
||||
get(id: string): Promise<string | null>;
|
||||
getWithMetadata<T>(id: string): Promise<{ value: string | null; metadata: T | null }>;
|
||||
delete(id: string): Promise<void>;
|
||||
list(options?: MetadataListOptions): Promise<MetadataListResult>;
|
||||
}
|
||||
|
||||
interface MetadataPutOptions {
|
||||
expiration?: number;
|
||||
expirationTtl?: number;
|
||||
metadata?: unknown;
|
||||
}
|
||||
|
||||
interface MetadataListOptions {
|
||||
prefix?: string | null;
|
||||
cursor?: string | null;
|
||||
limit?: number;
|
||||
}
|
||||
|
||||
interface MetadataListResult {
|
||||
keys: { name: string; expiration?: number }[];
|
||||
list_complete: boolean;
|
||||
cursor?: string;
|
||||
}
|
||||
|
||||
interface UploadQueueShim<Body = unknown> {
|
||||
send(body: Body, options?: UploadSendOptions): Promise<void>;
|
||||
sendBatch(bodies: Iterable<UploadSendRequest<Body>>): Promise<void>;
|
||||
}
|
||||
|
||||
interface UploadSendOptions {
|
||||
contentType?: UploadContentType;
|
||||
delaySeconds?: number;
|
||||
}
|
||||
|
||||
type UploadContentType = 'text' | 'bytes' | 'json' | 'v8';
|
||||
|
||||
interface UploadSendRequest<Body = unknown> {
|
||||
body: Body;
|
||||
options?: UploadSendOptions;
|
||||
}
|
||||
|
||||
interface UploadMessageShim<Body = unknown> {
|
||||
readonly id: string;
|
||||
readonly timestamp: Date;
|
||||
readonly body: Body;
|
||||
readonly attempts: number;
|
||||
retry(options?: UploadRetryOptions): void;
|
||||
ack(): void;
|
||||
}
|
||||
|
||||
interface UploadRetryOptions {
|
||||
delaySeconds?: number;
|
||||
}
|
||||
|
||||
interface UploadMessageBatch<Body = unknown> {
|
||||
readonly messages: readonly UploadMessageShim<Body>[];
|
||||
readonly queue: string;
|
||||
retryAll(options?: UploadRetryOptions): void;
|
||||
ackAll(): void;
|
||||
}
|
||||
|
||||
interface StreamPipeOptionsShim {
|
||||
preventClose?: boolean;
|
||||
preventAbort?: boolean;
|
||||
preventCancel?: boolean;
|
||||
signal?: AbortSignal;
|
||||
}
|
||||
|
||||
interface ByteCounterShim {
|
||||
readonly transform: TransformStream<Uint8Array, Uint8Array>;
|
||||
total(): number;
|
||||
}
|
||||
|
||||
interface StreamLimitShim {
|
||||
readonly limit: number;
|
||||
readonly seen: number;
|
||||
exceeded(): boolean;
|
||||
}
|
||||
|
||||
interface RequestBodyShim {
|
||||
readonly body: ReadableStream<Uint8Array> | null;
|
||||
readonly bodyUsed: boolean;
|
||||
readonly headers: Headers;
|
||||
readonly url: string;
|
||||
arrayBuffer(): Promise<ArrayBuffer>;
|
||||
formData(): Promise<FormData>;
|
||||
blob(): Promise<Blob>;
|
||||
}
|
||||
|
||||
interface ParsedUploadShim {
|
||||
key: string;
|
||||
contentType: string;
|
||||
width: number;
|
||||
height: number;
|
||||
format: string;
|
||||
}
|
||||
|
||||
interface ImageTransformerShim {
|
||||
transform(transform: ImageTransformShim): ImageTransformerShim;
|
||||
output(options: ImageOutputShim): Promise<ImageResultShim>;
|
||||
}
|
||||
|
||||
interface ImageTransformShim {
|
||||
width?: number;
|
||||
height?: number;
|
||||
fit?: 'scale-down' | 'contain' | 'cover' | 'crop' | 'pad';
|
||||
rotate?: number;
|
||||
}
|
||||
|
||||
interface ImageOutputShim {
|
||||
format?: string;
|
||||
quality?: number;
|
||||
background?: string;
|
||||
}
|
||||
|
||||
interface ImageResultShim {
|
||||
contentType(): string;
|
||||
image(): ReadableStream<Uint8Array>;
|
||||
response(): Response;
|
||||
}
|
||||
|
||||
interface UploadEnvShim {
|
||||
UPLOADS: UploadStorage;
|
||||
METADATA: MetadataStoreShim;
|
||||
UPLOAD_QUEUE: UploadQueueShim<unknown>;
|
||||
}
|
||||
}
|
||||
|
||||
export {};
|
||||
@@ -0,0 +1,271 @@
|
||||
// Generated by Wrangler by running `wrangler types` (hash: 4f1c8ad2b90e)
|
||||
// Runtime types generated with workerd@1.20260701.0 2026-07-01 nodejs_compat
|
||||
declare namespace Cloudflare {
|
||||
interface Env {
|
||||
UPLOADS: R2Bucket;
|
||||
METADATA: KVNamespace;
|
||||
UPLOAD_QUEUE: Queue<UploadMessageBody>;
|
||||
IMAGES: ImagesBinding;
|
||||
}
|
||||
}
|
||||
|
||||
interface UploadMessageBody {
|
||||
key: string;
|
||||
metadataId: string;
|
||||
contentType: string;
|
||||
}
|
||||
|
||||
interface R2Bucket {
|
||||
head(key: string): Promise<R2Object | null>;
|
||||
get(key: string, options?: R2GetOptions): Promise<R2ObjectBody | null>;
|
||||
put(
|
||||
key: string,
|
||||
value: ReadableStream | ArrayBuffer | string | null,
|
||||
options?: R2PutOptions,
|
||||
): Promise<R2Object>;
|
||||
delete(keys: string | string[]): Promise<void>;
|
||||
list(options?: R2ListOptions): Promise<R2Objects>;
|
||||
createMultipartUpload(key: string, options?: R2MultipartOptions): Promise<R2MultipartUpload>;
|
||||
resumeMultipartUpload(key: string, uploadId: string): R2MultipartUpload;
|
||||
}
|
||||
|
||||
interface R2Object {
|
||||
readonly key: string;
|
||||
readonly version: string;
|
||||
readonly size: number;
|
||||
readonly etag: string;
|
||||
readonly httpEtag: string;
|
||||
readonly checksums: R2Checksums;
|
||||
readonly uploaded: Date;
|
||||
readonly httpMetadata?: R2HTTPMetadata;
|
||||
readonly customMetadata?: Record<string, string>;
|
||||
readonly range?: R2Range;
|
||||
readonly storageClass: string;
|
||||
writeHttpMetadata(headers: Headers): void;
|
||||
}
|
||||
|
||||
interface R2ObjectBody extends R2Object {
|
||||
get body(): ReadableStream;
|
||||
get bodyUsed(): boolean;
|
||||
arrayBuffer(): Promise<ArrayBuffer>;
|
||||
text(): Promise<string>;
|
||||
json<T>(): Promise<T>;
|
||||
blob(): Promise<Blob>;
|
||||
bytes(): Promise<Uint8Array>;
|
||||
}
|
||||
|
||||
interface R2GetOptions {
|
||||
onlyIf?: R2Conditional | Headers;
|
||||
range?: R2Range;
|
||||
ssecKey?: ArrayBuffer | string;
|
||||
}
|
||||
|
||||
interface R2PutOptions {
|
||||
onlyIf?: R2Conditional | Headers;
|
||||
httpMetadata?: R2HTTPMetadata | Headers;
|
||||
customMetadata?: Record<string, string>;
|
||||
md5?: ArrayBuffer | string;
|
||||
sha1?: ArrayBuffer | string;
|
||||
sha256?: ArrayBuffer | string;
|
||||
storageClass?: string;
|
||||
ssecKey?: ArrayBuffer | string;
|
||||
}
|
||||
|
||||
interface R2ListOptions {
|
||||
limit?: number;
|
||||
prefix?: string;
|
||||
cursor?: string;
|
||||
delimiter?: string;
|
||||
startAfter?: string;
|
||||
include?: ('httpMetadata' | 'customMetadata')[];
|
||||
}
|
||||
|
||||
interface R2Objects {
|
||||
objects: R2Object[];
|
||||
truncated: boolean;
|
||||
cursor?: string;
|
||||
delimitedPrefixes: string[];
|
||||
}
|
||||
|
||||
interface R2MultipartOptions {
|
||||
httpMetadata?: R2HTTPMetadata | Headers;
|
||||
customMetadata?: Record<string, string>;
|
||||
storageClass?: string;
|
||||
}
|
||||
|
||||
interface R2MultipartUpload {
|
||||
readonly key: string;
|
||||
readonly uploadId: string;
|
||||
uploadPart(
|
||||
partNumber: number,
|
||||
value: ReadableStream | ArrayBuffer | string | Blob,
|
||||
): Promise<R2UploadedPart>;
|
||||
abort(): Promise<void>;
|
||||
complete(uploadedParts: R2UploadedPart[]): Promise<R2Object>;
|
||||
}
|
||||
|
||||
interface R2UploadedPart {
|
||||
partNumber: number;
|
||||
etag: string;
|
||||
}
|
||||
|
||||
interface R2HTTPMetadata {
|
||||
contentType?: string;
|
||||
contentLanguage?: string;
|
||||
contentDisposition?: string;
|
||||
contentEncoding?: string;
|
||||
cacheControl?: string;
|
||||
cacheExpiry?: Date;
|
||||
}
|
||||
|
||||
interface R2Checksums {
|
||||
readonly md5?: ArrayBuffer;
|
||||
readonly sha1?: ArrayBuffer;
|
||||
readonly sha256?: ArrayBuffer;
|
||||
toJSON(): R2StringChecksums;
|
||||
}
|
||||
|
||||
interface R2StringChecksums {
|
||||
md5?: string;
|
||||
sha1?: string;
|
||||
sha256?: string;
|
||||
}
|
||||
|
||||
interface R2Conditional {
|
||||
etagMatches?: string;
|
||||
etagDoesNotMatch?: string;
|
||||
uploadedBefore?: Date;
|
||||
uploadedAfter?: Date;
|
||||
secondsGranularity?: boolean;
|
||||
}
|
||||
|
||||
interface R2Range {
|
||||
offset?: number;
|
||||
length?: number;
|
||||
suffix?: number;
|
||||
}
|
||||
|
||||
interface KVNamespace<Key extends string = string> {
|
||||
get(key: Key, options?: Partial<KVNamespaceGetOptions<undefined>>): Promise<string | null>;
|
||||
getWithMetadata<Metadata = unknown>(
|
||||
key: Key,
|
||||
options?: Partial<KVNamespaceGetOptions<undefined>>,
|
||||
): Promise<KVNamespaceGetWithMetadataResult<string, Metadata>>;
|
||||
put(
|
||||
key: Key,
|
||||
value: string | ArrayBuffer | ArrayBufferView | ReadableStream,
|
||||
options?: KVNamespacePutOptions,
|
||||
): Promise<void>;
|
||||
delete(key: Key): Promise<void>;
|
||||
list<Metadata = unknown>(
|
||||
options?: KVNamespaceListOptions,
|
||||
): Promise<KVNamespaceListResult<Metadata, Key>>;
|
||||
}
|
||||
|
||||
interface KVNamespaceGetOptions<Type> {
|
||||
type: Type;
|
||||
cacheTtl?: number;
|
||||
}
|
||||
|
||||
interface KVNamespacePutOptions {
|
||||
expiration?: number;
|
||||
expirationTtl?: number;
|
||||
metadata?: unknown | null;
|
||||
}
|
||||
|
||||
interface KVNamespaceListOptions {
|
||||
limit?: number;
|
||||
prefix?: string | null;
|
||||
cursor?: string | null;
|
||||
}
|
||||
|
||||
interface KVNamespaceListResult<Metadata, Key extends string = string> {
|
||||
keys: KVNamespaceListKey<Metadata, Key>[];
|
||||
list_complete: boolean;
|
||||
cursor?: string;
|
||||
}
|
||||
|
||||
interface KVNamespaceListKey<Metadata, Key extends string = string> {
|
||||
name: Key;
|
||||
expiration?: number;
|
||||
metadata?: Metadata;
|
||||
}
|
||||
|
||||
interface KVNamespaceGetWithMetadataResult<Value, Metadata> {
|
||||
value: Value | null;
|
||||
metadata: Metadata | null;
|
||||
cacheStatus: string | null;
|
||||
}
|
||||
|
||||
interface Queue<Body = unknown> {
|
||||
send(message: Body, options?: QueueSendOptions): Promise<void>;
|
||||
sendBatch(messages: Iterable<MessageSendRequest<Body>>): Promise<void>;
|
||||
}
|
||||
|
||||
interface QueueSendOptions {
|
||||
contentType?: QueueContentType;
|
||||
delaySeconds?: number;
|
||||
}
|
||||
|
||||
type QueueContentType = 'text' | 'bytes' | 'json' | 'v8';
|
||||
|
||||
interface MessageSendRequest<Body = unknown> {
|
||||
body: Body;
|
||||
options?: QueueSendOptions;
|
||||
}
|
||||
|
||||
interface Message<Body = unknown> {
|
||||
readonly id: string;
|
||||
readonly timestamp: Date;
|
||||
readonly body: Body;
|
||||
readonly attempts: number;
|
||||
retry(options?: QueueRetryOptions): void;
|
||||
ack(): void;
|
||||
}
|
||||
|
||||
interface QueueRetryOptions {
|
||||
delaySeconds?: number;
|
||||
}
|
||||
|
||||
interface MessageBatch<Body = unknown> {
|
||||
readonly messages: readonly Message<Body>[];
|
||||
readonly queue: string;
|
||||
retryAll(options?: QueueRetryOptions): void;
|
||||
ackAll(): void;
|
||||
}
|
||||
|
||||
interface ImagesBinding {
|
||||
info(stream: ReadableStream<Uint8Array>): Promise<ImageMetadata>;
|
||||
input(stream: ReadableStream<Uint8Array>): ImageTransformer;
|
||||
}
|
||||
|
||||
interface ImageMetadata {
|
||||
format: string;
|
||||
fileSize: number;
|
||||
width: number;
|
||||
height: number;
|
||||
}
|
||||
|
||||
interface ImageTransformer {
|
||||
transform(transform: ImageTransform): ImageTransformer;
|
||||
output(options: ImageOutputOptions): Promise<ImageTransformationResult>;
|
||||
}
|
||||
|
||||
interface ImageTransform {
|
||||
width?: number;
|
||||
height?: number;
|
||||
fit?: 'scale-down' | 'contain' | 'cover' | 'crop' | 'pad';
|
||||
rotate?: number;
|
||||
}
|
||||
|
||||
interface ImageOutputOptions {
|
||||
format?: string;
|
||||
quality?: number;
|
||||
background?: string;
|
||||
}
|
||||
|
||||
interface ImageTransformationResult {
|
||||
contentType(): string;
|
||||
image(): ReadableStream<Uint8Array>;
|
||||
response(): Response;
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"name": "dense-header-fixture",
|
||||
"private": true,
|
||||
"version": "0.0.0",
|
||||
"type": "module"
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
import type { URLSessionTask } from './types';
|
||||
|
||||
export class RequestQueue {
|
||||
private readonly waiting: URLSessionTask[] = [];
|
||||
private running = 0;
|
||||
|
||||
enqueue(task: URLSessionTask, limit: number): void {
|
||||
if (this.running < limit) {
|
||||
this.running += 1;
|
||||
return;
|
||||
}
|
||||
this.waiting.push(task);
|
||||
}
|
||||
|
||||
release(): URLSessionTask | undefined {
|
||||
this.running = Math.max(0, this.running - 1);
|
||||
return this.waiting.shift();
|
||||
}
|
||||
|
||||
get depth(): number {
|
||||
return this.waiting.length;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
import type { CachePolicy, URLRequest } from './types';
|
||||
|
||||
export function buildURLRequest(options: {
|
||||
url: string;
|
||||
method: string;
|
||||
body?: Uint8Array;
|
||||
headers: Record<string, string>;
|
||||
timeout: number;
|
||||
cachePolicy: CachePolicy;
|
||||
}): URLRequest {
|
||||
const headers = { ...options.headers };
|
||||
if (options.body && !headers['content-length']) {
|
||||
headers['content-length'] = String(options.body.length);
|
||||
}
|
||||
return {
|
||||
url: normalize(options.url),
|
||||
method: options.method.toUpperCase(),
|
||||
headers,
|
||||
body: options.body,
|
||||
timeout: options.timeout,
|
||||
cachePolicy: options.cachePolicy,
|
||||
};
|
||||
}
|
||||
|
||||
function normalize(url: string): string {
|
||||
return url.endsWith('/') && url.split('/').length > 4 ? url.slice(0, -1) : url;
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
import type { RequestDelegate, TaskResponse, URLRequest, URLSessionTask } from './types';
|
||||
|
||||
export function makeTask(options: {
|
||||
identifier: number;
|
||||
request: URLRequest;
|
||||
delegate: RequestDelegate;
|
||||
allowsCellularAccess: boolean;
|
||||
waitsForConnectivity: boolean;
|
||||
resourceTimeout: number;
|
||||
}): URLSessionTask {
|
||||
const handlers: Array<(response: TaskResponse) => void> = [];
|
||||
return {
|
||||
identifier: options.identifier,
|
||||
request: options.request,
|
||||
state: 'initialized',
|
||||
cancel() { this.state = 'cancelled'; },
|
||||
onComplete(handler) { handlers.push(handler); },
|
||||
};
|
||||
}
|
||||
|
||||
export function resumeTask(task: URLSessionTask): void {
|
||||
task.state = 'resumed';
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
export type CachePolicy = 'useProtocolCachePolicy' | 'reloadIgnoringLocalCacheData' | 'returnCacheDataElseLoad';
|
||||
export type RequestState = 'initialized' | 'resumed' | 'suspended' | 'cancelled' | 'finished';
|
||||
|
||||
export interface URLRequest {
|
||||
url: string;
|
||||
method: string;
|
||||
headers: Record<string, string>;
|
||||
body?: Uint8Array;
|
||||
timeout: number;
|
||||
cachePolicy: CachePolicy;
|
||||
}
|
||||
|
||||
export interface TaskResponse {
|
||||
status: number;
|
||||
headers: Record<string, string>;
|
||||
body: Uint8Array;
|
||||
}
|
||||
|
||||
export interface URLSessionTask {
|
||||
identifier: number;
|
||||
request: URLRequest;
|
||||
state: RequestState;
|
||||
cancel(): void;
|
||||
onComplete(handler: (response: TaskResponse) => void): void;
|
||||
}
|
||||
|
||||
export interface Adapter { adapt(request: URLRequest): URLRequest; }
|
||||
export interface Serializer { serialize(value: unknown): Uint8Array; }
|
||||
export interface Validator { validate(response: TaskResponse): { ok: boolean; reason?: string }; }
|
||||
export interface Retrier { shouldRetry(response: TaskResponse, verdict: { ok: boolean }): boolean; }
|
||||
export interface RedirectHandler { resolve(location: string, original: URLRequest): { url: string; method: string; body?: Uint8Array } | null; }
|
||||
export interface TrustEvaluator { evaluate(host: string): boolean; }
|
||||
export interface Credential { apply(request: URLRequest): URLRequest; }
|
||||
export interface Interceptor { name: string; adapt(request: URLRequest, session: unknown): Promise<URLRequest>; }
|
||||
export interface RequestDelegate { willSend(request: URLRequest): void; }
|
||||
export interface EventMonitor {
|
||||
didAdaptRequest(request: URLRequest, interceptor: string): void;
|
||||
didCreateTask(task: URLSessionTask, request: URLRequest): void;
|
||||
didResumeTask(task: URLSessionTask): void;
|
||||
didRetryTask(task: URLSessionTask, previousIdentifier: number): void;
|
||||
didCompleteTask(task: URLSessionTask, response: TaskResponse): void;
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
export { Session } from './net/session';
|
||||
export { RequestQueue } from './core/queue';
|
||||
export { buildURLRequest } from './core/request-builder';
|
||||
@@ -0,0 +1,285 @@
|
||||
import type {
|
||||
Adapter,
|
||||
CachePolicy,
|
||||
Credential,
|
||||
EventMonitor,
|
||||
Interceptor,
|
||||
RedirectHandler,
|
||||
RequestDelegate,
|
||||
RequestState,
|
||||
Retrier,
|
||||
Serializer,
|
||||
TrustEvaluator,
|
||||
URLRequest,
|
||||
URLSessionTask,
|
||||
Validator,
|
||||
} from '../core/types';
|
||||
import { buildURLRequest } from '../core/request-builder';
|
||||
import { makeTask, resumeTask } from '../core/task-factory';
|
||||
import { RequestQueue } from '../core/queue';
|
||||
|
||||
/**
|
||||
* The shape density-first ranking exists for: a class whose top-of-file header
|
||||
* is a long, tightly-packed property list — dozens of adjacent declarations,
|
||||
* each individually trivial — while the methods a flow question actually asks
|
||||
* about live hundreds of lines below it.
|
||||
*
|
||||
* Ranked by density alone the header wins the file's whole budget and the
|
||||
* methods are buried. The ranking puts importance first for exactly this
|
||||
* reason, and density only breaks ties inside one importance tier.
|
||||
*/
|
||||
export class Session {
|
||||
readonly identifier: string;
|
||||
readonly adapter: Adapter;
|
||||
readonly serializer: Serializer;
|
||||
readonly validator: Validator;
|
||||
readonly retrier: Retrier;
|
||||
readonly redirectHandler: RedirectHandler;
|
||||
readonly trustEvaluator: TrustEvaluator;
|
||||
readonly eventMonitor: EventMonitor;
|
||||
readonly cachePolicy: CachePolicy;
|
||||
readonly credential: Credential | null;
|
||||
readonly interceptors: Interceptor[];
|
||||
readonly delegate: RequestDelegate;
|
||||
readonly queue: RequestQueue;
|
||||
readonly startRequestsImmediately: boolean;
|
||||
readonly maximumConnectionsPerHost: number;
|
||||
readonly timeoutIntervalForRequest: number;
|
||||
readonly timeoutIntervalForResource: number;
|
||||
readonly allowsCellularAccess: boolean;
|
||||
readonly waitsForConnectivity: boolean;
|
||||
readonly httpShouldUsePipelining: boolean;
|
||||
readonly httpShouldSetCookies: boolean;
|
||||
readonly httpMaximumConnectionsPerHost: number;
|
||||
readonly sessionConfigurationName: string;
|
||||
readonly requestState: RequestState;
|
||||
readonly defaultHeaders: Record<string, string>;
|
||||
readonly userAgent: string;
|
||||
readonly acceptEncoding: string;
|
||||
readonly acceptLanguage: string;
|
||||
private taskCounter = 0;
|
||||
private active = new Map<number, URLSessionTask>();
|
||||
|
||||
constructor(options: Partial<Session> & { identifier: string }) {
|
||||
this.identifier = options.identifier;
|
||||
this.adapter = options.adapter!;
|
||||
this.serializer = options.serializer!;
|
||||
this.validator = options.validator!;
|
||||
this.retrier = options.retrier!;
|
||||
this.redirectHandler = options.redirectHandler!;
|
||||
this.trustEvaluator = options.trustEvaluator!;
|
||||
this.eventMonitor = options.eventMonitor!;
|
||||
this.cachePolicy = options.cachePolicy ?? 'useProtocolCachePolicy';
|
||||
this.credential = options.credential ?? null;
|
||||
this.interceptors = options.interceptors ?? [];
|
||||
this.delegate = options.delegate!;
|
||||
this.queue = options.queue ?? new RequestQueue();
|
||||
this.startRequestsImmediately = options.startRequestsImmediately ?? true;
|
||||
this.maximumConnectionsPerHost = options.maximumConnectionsPerHost ?? 6;
|
||||
this.timeoutIntervalForRequest = options.timeoutIntervalForRequest ?? 60;
|
||||
this.timeoutIntervalForResource = options.timeoutIntervalForResource ?? 604800;
|
||||
this.allowsCellularAccess = options.allowsCellularAccess ?? true;
|
||||
this.waitsForConnectivity = options.waitsForConnectivity ?? false;
|
||||
this.httpShouldUsePipelining = options.httpShouldUsePipelining ?? false;
|
||||
this.httpShouldSetCookies = options.httpShouldSetCookies ?? true;
|
||||
this.httpMaximumConnectionsPerHost = options.httpMaximumConnectionsPerHost ?? 6;
|
||||
this.sessionConfigurationName = options.sessionConfigurationName ?? 'default';
|
||||
this.requestState = options.requestState ?? 'initialized';
|
||||
this.defaultHeaders = options.defaultHeaders ?? {};
|
||||
this.userAgent = options.userAgent ?? 'session/1.0';
|
||||
this.acceptEncoding = options.acceptEncoding ?? 'br;q=1.0, gzip;q=0.9';
|
||||
this.acceptLanguage = options.acceptLanguage ?? 'en;q=1.0';
|
||||
}
|
||||
|
||||
// -- configuration accessors ----------------------------------------------
|
||||
// Individually trivial, adjacent, and dense. On the density tiebreak alone
|
||||
// this block outranks anything with a body worth reading.
|
||||
|
||||
get isBackground(): boolean {
|
||||
return this.sessionConfigurationName === 'background';
|
||||
}
|
||||
|
||||
get connectionLimit(): number {
|
||||
return Math.min(this.maximumConnectionsPerHost, this.httpMaximumConnectionsPerHost);
|
||||
}
|
||||
|
||||
get headerDefaults(): Record<string, string> {
|
||||
return { ...this.defaultHeaders, 'user-agent': this.userAgent };
|
||||
}
|
||||
|
||||
get acceptHeaders(): Record<string, string> {
|
||||
return { 'accept-encoding': this.acceptEncoding, 'accept-language': this.acceptLanguage };
|
||||
}
|
||||
|
||||
get activeCount(): number {
|
||||
return this.active.size;
|
||||
}
|
||||
|
||||
get isIdle(): boolean {
|
||||
return this.active.size === 0;
|
||||
}
|
||||
|
||||
get nextIdentifier(): number {
|
||||
return this.taskCounter + 1;
|
||||
}
|
||||
|
||||
get description(): string {
|
||||
return `Session(${this.identifier}, ${this.sessionConfigurationName})`;
|
||||
}
|
||||
|
||||
cancelAll(): void {
|
||||
for (const task of this.active.values()) task.cancel();
|
||||
this.active.clear();
|
||||
}
|
||||
|
||||
taskFor(identifier: number): URLSessionTask | undefined {
|
||||
return this.active.get(identifier);
|
||||
}
|
||||
|
||||
headers(): Record<string, string> {
|
||||
return { ...this.headerDefaults, ...this.acceptHeaders };
|
||||
}
|
||||
|
||||
withUserAgent(userAgent: string): Session {
|
||||
return new Session({ ...this, identifier: this.identifier, userAgent });
|
||||
}
|
||||
|
||||
withTimeout(seconds: number): Session {
|
||||
return new Session({ ...this, identifier: this.identifier, timeoutIntervalForRequest: seconds });
|
||||
}
|
||||
|
||||
withInterceptor(interceptor: Interceptor): Session {
|
||||
return new Session({
|
||||
...this,
|
||||
identifier: this.identifier,
|
||||
interceptors: [...this.interceptors, interceptor],
|
||||
});
|
||||
}
|
||||
|
||||
withCredential(credential: Credential): Session {
|
||||
return new Session({ ...this, identifier: this.identifier, credential });
|
||||
}
|
||||
|
||||
withCachePolicy(cachePolicy: CachePolicy): Session {
|
||||
return new Session({ ...this, identifier: this.identifier, cachePolicy });
|
||||
}
|
||||
|
||||
withQueue(queue: RequestQueue): Session {
|
||||
return new Session({ ...this, identifier: this.identifier, queue });
|
||||
}
|
||||
|
||||
withAdapter(adapter: Adapter): Session {
|
||||
return new Session({ ...this, identifier: this.identifier, adapter });
|
||||
}
|
||||
|
||||
withValidator(validator: Validator): Session {
|
||||
return new Session({ ...this, identifier: this.identifier, validator });
|
||||
}
|
||||
|
||||
withRetrier(retrier: Retrier): Session {
|
||||
return new Session({ ...this, identifier: this.identifier, retrier });
|
||||
}
|
||||
|
||||
withMonitor(eventMonitor: EventMonitor): Session {
|
||||
return new Session({ ...this, identifier: this.identifier, eventMonitor });
|
||||
}
|
||||
|
||||
// -- the flow ---------------------------------------------------------------
|
||||
//
|
||||
// The methods below are what a "how does a request get built and sent" question
|
||||
// is about, and they sit hundreds of lines under the header block.
|
||||
|
||||
/**
|
||||
* Turn a convenience call into a URLRequest, hand it to the adapter chain and
|
||||
* start the resulting task. The entry point of the whole flow.
|
||||
*/
|
||||
async perform(url: string, method: string, body?: Uint8Array): Promise<URLSessionTask> {
|
||||
const initial = buildURLRequest({
|
||||
url,
|
||||
method,
|
||||
body,
|
||||
headers: this.headers(),
|
||||
timeout: this.timeoutIntervalForRequest,
|
||||
cachePolicy: this.cachePolicy,
|
||||
});
|
||||
const adapted = await this.adapt(initial);
|
||||
return this.didCreateURLRequest(adapted);
|
||||
}
|
||||
|
||||
/**
|
||||
* Every interceptor gets a chance to rewrite the request before it becomes a
|
||||
* task. Runs in registration order, and a thrown error aborts the whole call.
|
||||
*/
|
||||
private async adapt(request: URLRequest): Promise<URLRequest> {
|
||||
let current = request;
|
||||
for (const interceptor of this.interceptors) {
|
||||
current = await interceptor.adapt(current, this);
|
||||
this.eventMonitor.didAdaptRequest(current, interceptor.name);
|
||||
}
|
||||
if (this.credential) current = this.credential.apply(current);
|
||||
return current;
|
||||
}
|
||||
|
||||
/**
|
||||
* The adapted request is final: build the task around it, register it and —
|
||||
* unless the session was told to wait — resume it immediately.
|
||||
*/
|
||||
didCreateURLRequest(request: URLRequest): URLSessionTask {
|
||||
this.taskCounter += 1;
|
||||
const identifier = this.taskCounter;
|
||||
const created = this.task(request, identifier);
|
||||
this.active.set(identifier, created);
|
||||
this.eventMonitor.didCreateTask(created, request);
|
||||
if (this.startRequestsImmediately) this.resume(created);
|
||||
return created;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the URLSessionTask for a request. Split out from
|
||||
* `didCreateURLRequest` because retries rebuild the task without going back
|
||||
* through the adapter chain.
|
||||
*/
|
||||
task(request: URLRequest, identifier: number): URLSessionTask {
|
||||
const created = makeTask({
|
||||
identifier,
|
||||
request,
|
||||
delegate: this.delegate,
|
||||
allowsCellularAccess: this.allowsCellularAccess,
|
||||
waitsForConnectivity: this.waitsForConnectivity,
|
||||
resourceTimeout: this.timeoutIntervalForResource,
|
||||
});
|
||||
created.onComplete((response) => {
|
||||
this.active.delete(identifier);
|
||||
const verdict = this.validator.validate(response);
|
||||
if (!verdict.ok && this.retrier.shouldRetry(response, verdict)) {
|
||||
this.retry(request, identifier);
|
||||
return;
|
||||
}
|
||||
this.eventMonitor.didCompleteTask(created, response);
|
||||
});
|
||||
return created;
|
||||
}
|
||||
|
||||
/** Put a built task on the queue and start it. */
|
||||
resume(task: URLSessionTask): void {
|
||||
this.queue.enqueue(task, this.connectionLimit);
|
||||
resumeTask(task);
|
||||
this.eventMonitor.didResumeTask(task);
|
||||
}
|
||||
|
||||
/** Rebuild and restart a task the retrier asked for. */
|
||||
private retry(request: URLRequest, previousIdentifier: number): void {
|
||||
this.taskCounter += 1;
|
||||
const retried = this.task(request, this.taskCounter);
|
||||
this.active.set(this.taskCounter, retried);
|
||||
this.eventMonitor.didRetryTask(retried, previousIdentifier);
|
||||
this.resume(retried);
|
||||
}
|
||||
|
||||
/** Follow a redirect by adapting and re-performing the new location. */
|
||||
async follow(response: { location: string }, original: URLRequest): Promise<URLSessionTask> {
|
||||
const target = this.redirectHandler.resolve(response.location, original);
|
||||
if (!target) throw new Error(`redirect to ${response.location} refused`);
|
||||
return this.perform(target.url, target.method, target.body);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"name": "displacement-fixture",
|
||||
"version": "1.0.0",
|
||||
"private": true,
|
||||
"type": "module"
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
import { ingestRecords } from './pipeline/ingest';
|
||||
import { normalizeRecords } from './pipeline/normalize';
|
||||
import { enrichRecords } from './pipeline/enrich';
|
||||
import { publishRecords } from './pipeline/publish';
|
||||
import type { PipelineOptions, PipelineRecord, RawRecord } from './pipeline/types';
|
||||
|
||||
/** Run one batch through every pipeline stage, in order. */
|
||||
export function runPipeline(batch: RawRecord[], options: PipelineOptions): PipelineRecord[] {
|
||||
return publishRecords(enrichRecords(normalizeRecords(ingestRecords(batch, options), options), options), options);
|
||||
}
|
||||
@@ -0,0 +1,127 @@
|
||||
import { writeBatch } from './sink';
|
||||
import type { PipelineOptions, PipelineRecord } from './types';
|
||||
|
||||
/** Enrich every record in a batch. */
|
||||
export function enrichRecords(records: PipelineRecord[], options: PipelineOptions): PipelineRecord[] {
|
||||
const out: PipelineRecord[] = [];
|
||||
for (const record of records) {
|
||||
const tags = [...record.tags];
|
||||
const warnings = [...record.warnings];
|
||||
let value = record.value;
|
||||
|
||||
// 1. segment
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('segment:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('segment: missing after enrich');
|
||||
} else {
|
||||
value = weightFacet(value, hit.length);
|
||||
tags.push('segment.enri');
|
||||
}
|
||||
}
|
||||
|
||||
// 2. referrer
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('referrer:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('referrer: missing after enrich');
|
||||
} else {
|
||||
value = blendFacet(value, hit.length);
|
||||
tags.push('referrer.enri');
|
||||
}
|
||||
}
|
||||
|
||||
// 3. experiment
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('experiment:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('experiment: missing after enrich');
|
||||
} else {
|
||||
value = weightFacet(value, hit.length);
|
||||
tags.push('experiment.enri');
|
||||
}
|
||||
}
|
||||
|
||||
// 4. subscription
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('subscription:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('subscription: missing after enrich');
|
||||
} else {
|
||||
value = blendFacet(value, hit.length);
|
||||
tags.push('subscription.enri');
|
||||
}
|
||||
}
|
||||
|
||||
// 5. entitlement
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('entitlement:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('entitlement: missing after enrich');
|
||||
} else {
|
||||
value = weightFacet(value, hit.length);
|
||||
tags.push('entitlement.enri');
|
||||
}
|
||||
}
|
||||
|
||||
// 6. invoice
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('invoice:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('invoice: missing after enrich');
|
||||
} else {
|
||||
value = blendFacet(value, hit.length);
|
||||
tags.push('invoice.enri');
|
||||
}
|
||||
}
|
||||
|
||||
// 7. refund
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('refund:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('refund: missing after enrich');
|
||||
} else {
|
||||
value = weightFacet(value, hit.length);
|
||||
tags.push('refund.enri');
|
||||
}
|
||||
}
|
||||
|
||||
// 8. dispute
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('dispute:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('dispute: missing after enrich');
|
||||
} else {
|
||||
value = blendFacet(value, hit.length);
|
||||
tags.push('dispute.enri');
|
||||
}
|
||||
}
|
||||
|
||||
// 9. payout
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('payout:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('payout: missing after enrich');
|
||||
} else {
|
||||
value = weightFacet(value, hit.length);
|
||||
tags.push('payout.enri');
|
||||
}
|
||||
}
|
||||
|
||||
out.push({ ...record, value, tags: tags.slice(0, options.maxTags), warnings });
|
||||
}
|
||||
writeBatch('enrichRecords', out);
|
||||
return out;
|
||||
}
|
||||
|
||||
/** weightFacet — a small deterministic helper. */
|
||||
export function weightFacet(base: number, width: number): number {
|
||||
const scaled = base + width * 3 - (width % 7);
|
||||
return scaled < 0 ? 0 : scaled;
|
||||
}
|
||||
|
||||
/** blendFacet — a small deterministic helper. */
|
||||
export function blendFacet(base: number, width: number): number {
|
||||
const scaled = base + width * 3 - (width % 7);
|
||||
return scaled < 0 ? 0 : scaled;
|
||||
}
|
||||
@@ -0,0 +1,541 @@
|
||||
import { scaleFacet, clampFacet } from './normalize';
|
||||
import { writeBatch } from './sink';
|
||||
import type { PipelineOptions, PipelineRecord, RawRecord } from './types';
|
||||
|
||||
/**
|
||||
* Ingest one batch of raw records.
|
||||
*
|
||||
* Every facet is unpacked in its own block so an on-call engineer can read the
|
||||
* ingest end-to-end in one place. The shape is deliberately flat: this single
|
||||
* function is the whole stage, which is exactly the shape that makes it the
|
||||
* biggest cluster member in the file.
|
||||
*/
|
||||
export function ingestRecords(batch: RawRecord[], options: PipelineOptions): PipelineRecord[] {
|
||||
const out: PipelineRecord[] = [];
|
||||
for (const record of batch) {
|
||||
const tags: string[] = [];
|
||||
const warnings: string[] = [];
|
||||
let value = 0;
|
||||
|
||||
// 1. identity — normalise the identity facet of the record.
|
||||
{
|
||||
const raw = record.payload['identity'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('identity: empty, dropped');
|
||||
} else {
|
||||
const scaled = scaleFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('identity:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('identity: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 2. geography — normalise the geography facet of the record.
|
||||
{
|
||||
const raw = record.payload['geography'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('geography: empty, dropped');
|
||||
} else {
|
||||
const scaled = clampFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('geography:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('geography: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 3. currency — normalise the currency facet of the record.
|
||||
{
|
||||
const raw = record.payload['currency'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('currency: empty, dropped');
|
||||
} else {
|
||||
const scaled = scaleFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('currency:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('currency: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 4. timestamp — normalise the timestamp facet of the record.
|
||||
{
|
||||
const raw = record.payload['timestamp'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('timestamp: empty, dropped');
|
||||
} else {
|
||||
const scaled = clampFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('timestamp:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('timestamp: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 5. channel — normalise the channel facet of the record.
|
||||
{
|
||||
const raw = record.payload['channel'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('channel: empty, dropped');
|
||||
} else {
|
||||
const scaled = scaleFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('channel:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('channel: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 6. campaign — normalise the campaign facet of the record.
|
||||
{
|
||||
const raw = record.payload['campaign'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('campaign: empty, dropped');
|
||||
} else {
|
||||
const scaled = clampFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('campaign:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('campaign: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 7. device — normalise the device facet of the record.
|
||||
{
|
||||
const raw = record.payload['device'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('device: empty, dropped');
|
||||
} else {
|
||||
const scaled = scaleFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('device:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('device: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 8. locale — normalise the locale facet of the record.
|
||||
{
|
||||
const raw = record.payload['locale'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('locale: empty, dropped');
|
||||
} else {
|
||||
const scaled = clampFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('locale:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('locale: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 9. consent — normalise the consent facet of the record.
|
||||
{
|
||||
const raw = record.payload['consent'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('consent: empty, dropped');
|
||||
} else {
|
||||
const scaled = scaleFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('consent:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('consent: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 10. segment — normalise the segment facet of the record.
|
||||
{
|
||||
const raw = record.payload['segment'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('segment: empty, dropped');
|
||||
} else {
|
||||
const scaled = clampFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('segment:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('segment: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 11. referrer — normalise the referrer facet of the record.
|
||||
{
|
||||
const raw = record.payload['referrer'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('referrer: empty, dropped');
|
||||
} else {
|
||||
const scaled = scaleFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('referrer:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('referrer: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 12. experiment — normalise the experiment facet of the record.
|
||||
{
|
||||
const raw = record.payload['experiment'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('experiment: empty, dropped');
|
||||
} else {
|
||||
const scaled = clampFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('experiment:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('experiment: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 13. subscription — normalise the subscription facet of the record.
|
||||
{
|
||||
const raw = record.payload['subscription'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('subscription: empty, dropped');
|
||||
} else {
|
||||
const scaled = scaleFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('subscription:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('subscription: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 14. entitlement — normalise the entitlement facet of the record.
|
||||
{
|
||||
const raw = record.payload['entitlement'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('entitlement: empty, dropped');
|
||||
} else {
|
||||
const scaled = clampFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('entitlement:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('entitlement: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 15. invoice — normalise the invoice facet of the record.
|
||||
{
|
||||
const raw = record.payload['invoice'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('invoice: empty, dropped');
|
||||
} else {
|
||||
const scaled = scaleFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('invoice:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('invoice: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 16. refund — normalise the refund facet of the record.
|
||||
{
|
||||
const raw = record.payload['refund'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('refund: empty, dropped');
|
||||
} else {
|
||||
const scaled = clampFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('refund:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('refund: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 17. dispute — normalise the dispute facet of the record.
|
||||
{
|
||||
const raw = record.payload['dispute'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('dispute: empty, dropped');
|
||||
} else {
|
||||
const scaled = scaleFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('dispute:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('dispute: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 18. payout — normalise the payout facet of the record.
|
||||
{
|
||||
const raw = record.payload['payout'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('payout: empty, dropped');
|
||||
} else {
|
||||
const scaled = clampFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('payout:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('payout: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 19. shipment — normalise the shipment facet of the record.
|
||||
{
|
||||
const raw = record.payload['shipment'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('shipment: empty, dropped');
|
||||
} else {
|
||||
const scaled = scaleFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('shipment:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('shipment: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 20. inventory — normalise the inventory facet of the record.
|
||||
{
|
||||
const raw = record.payload['inventory'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('inventory: empty, dropped');
|
||||
} else {
|
||||
const scaled = clampFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('inventory:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('inventory: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 21. warehouse — normalise the warehouse facet of the record.
|
||||
{
|
||||
const raw = record.payload['warehouse'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('warehouse: empty, dropped');
|
||||
} else {
|
||||
const scaled = scaleFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('warehouse:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('warehouse: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 22. carrier — normalise the carrier facet of the record.
|
||||
{
|
||||
const raw = record.payload['carrier'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('carrier: empty, dropped');
|
||||
} else {
|
||||
const scaled = clampFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('carrier:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('carrier: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 23. customs — normalise the customs facet of the record.
|
||||
{
|
||||
const raw = record.payload['customs'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('customs: empty, dropped');
|
||||
} else {
|
||||
const scaled = scaleFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('customs:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('customs: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 24. tariff — normalise the tariff facet of the record.
|
||||
{
|
||||
const raw = record.payload['tariff'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('tariff: empty, dropped');
|
||||
} else {
|
||||
const scaled = clampFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('tariff:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('tariff: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 25. sensor — normalise the sensor facet of the record.
|
||||
{
|
||||
const raw = record.payload['sensor'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('sensor: empty, dropped');
|
||||
} else {
|
||||
const scaled = scaleFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('sensor:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('sensor: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 26. firmware — normalise the firmware facet of the record.
|
||||
{
|
||||
const raw = record.payload['firmware'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('firmware: empty, dropped');
|
||||
} else {
|
||||
const scaled = clampFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('firmware:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('firmware: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 27. telemetry — normalise the telemetry facet of the record.
|
||||
{
|
||||
const raw = record.payload['telemetry'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('telemetry: empty, dropped');
|
||||
} else {
|
||||
const scaled = scaleFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('telemetry:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('telemetry: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 28. battery — normalise the battery facet of the record.
|
||||
{
|
||||
const raw = record.payload['battery'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('battery: empty, dropped');
|
||||
} else {
|
||||
const scaled = clampFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('battery:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('battery: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 29. network — normalise the network facet of the record.
|
||||
{
|
||||
const raw = record.payload['network'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('network: empty, dropped');
|
||||
} else {
|
||||
const scaled = scaleFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('network:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('network: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 30. roaming — normalise the roaming facet of the record.
|
||||
{
|
||||
const raw = record.payload['roaming'];
|
||||
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
|
||||
if (text.length === 0 && options.dropEmpty) {
|
||||
warnings.push('roaming: empty, dropped');
|
||||
} else {
|
||||
const scaled = clampFacet(text.length, options.maxTags);
|
||||
if (Number.isFinite(scaled) && scaled !== 0) {
|
||||
tags.push('roaming:' + text.slice(0, 24));
|
||||
value += scaled;
|
||||
} else if (options.strict) {
|
||||
warnings.push('roaming: not scalable — ' + text.slice(0, 16));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
out.push({
|
||||
id: record.id,
|
||||
source: record.source,
|
||||
kind: options.defaultKind,
|
||||
value,
|
||||
tags: tags.slice(0, options.maxTags),
|
||||
warnings,
|
||||
});
|
||||
}
|
||||
writeBatch('ingest', out);
|
||||
return out;
|
||||
}
|
||||
@@ -0,0 +1,127 @@
|
||||
import { writeBatch } from './sink';
|
||||
import type { PipelineOptions, PipelineRecord } from './types';
|
||||
|
||||
/** Normalize every record in a batch. */
|
||||
export function normalizeRecords(records: PipelineRecord[], options: PipelineOptions): PipelineRecord[] {
|
||||
const out: PipelineRecord[] = [];
|
||||
for (const record of records) {
|
||||
const tags = [...record.tags];
|
||||
const warnings = [...record.warnings];
|
||||
let value = record.value;
|
||||
|
||||
// 1. identity
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('identity:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('identity: missing after normalize');
|
||||
} else {
|
||||
value = scaleFacet(value, hit.length);
|
||||
tags.push('identity.norm');
|
||||
}
|
||||
}
|
||||
|
||||
// 2. geography
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('geography:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('geography: missing after normalize');
|
||||
} else {
|
||||
value = clampFacet(value, hit.length);
|
||||
tags.push('geography.norm');
|
||||
}
|
||||
}
|
||||
|
||||
// 3. currency
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('currency:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('currency: missing after normalize');
|
||||
} else {
|
||||
value = scaleFacet(value, hit.length);
|
||||
tags.push('currency.norm');
|
||||
}
|
||||
}
|
||||
|
||||
// 4. timestamp
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('timestamp:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('timestamp: missing after normalize');
|
||||
} else {
|
||||
value = clampFacet(value, hit.length);
|
||||
tags.push('timestamp.norm');
|
||||
}
|
||||
}
|
||||
|
||||
// 5. channel
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('channel:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('channel: missing after normalize');
|
||||
} else {
|
||||
value = scaleFacet(value, hit.length);
|
||||
tags.push('channel.norm');
|
||||
}
|
||||
}
|
||||
|
||||
// 6. campaign
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('campaign:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('campaign: missing after normalize');
|
||||
} else {
|
||||
value = clampFacet(value, hit.length);
|
||||
tags.push('campaign.norm');
|
||||
}
|
||||
}
|
||||
|
||||
// 7. device
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('device:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('device: missing after normalize');
|
||||
} else {
|
||||
value = scaleFacet(value, hit.length);
|
||||
tags.push('device.norm');
|
||||
}
|
||||
}
|
||||
|
||||
// 8. locale
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('locale:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('locale: missing after normalize');
|
||||
} else {
|
||||
value = clampFacet(value, hit.length);
|
||||
tags.push('locale.norm');
|
||||
}
|
||||
}
|
||||
|
||||
// 9. consent
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('consent:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('consent: missing after normalize');
|
||||
} else {
|
||||
value = scaleFacet(value, hit.length);
|
||||
tags.push('consent.norm');
|
||||
}
|
||||
}
|
||||
|
||||
out.push({ ...record, value, tags: tags.slice(0, options.maxTags), warnings });
|
||||
}
|
||||
writeBatch('normalizeRecords', out);
|
||||
return out;
|
||||
}
|
||||
|
||||
/** scaleFacet — a small deterministic helper. */
|
||||
export function scaleFacet(base: number, width: number): number {
|
||||
const scaled = base + width * 3 - (width % 7);
|
||||
return scaled < 0 ? 0 : scaled;
|
||||
}
|
||||
|
||||
/** clampFacet — a small deterministic helper. */
|
||||
export function clampFacet(base: number, width: number): number {
|
||||
const scaled = base + width * 3 - (width % 7);
|
||||
return scaled < 0 ? 0 : scaled;
|
||||
}
|
||||
@@ -0,0 +1,127 @@
|
||||
import { writeBatch } from './sink';
|
||||
import type { PipelineOptions, PipelineRecord } from './types';
|
||||
|
||||
/** Publish every record in a batch. */
|
||||
export function publishRecords(records: PipelineRecord[], options: PipelineOptions): PipelineRecord[] {
|
||||
const out: PipelineRecord[] = [];
|
||||
for (const record of records) {
|
||||
const tags = [...record.tags];
|
||||
const warnings = [...record.warnings];
|
||||
let value = record.value;
|
||||
|
||||
// 1. shipment
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('shipment:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('shipment: missing after publish');
|
||||
} else {
|
||||
value = rankFacet(value, hit.length);
|
||||
tags.push('shipment.publ');
|
||||
}
|
||||
}
|
||||
|
||||
// 2. inventory
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('inventory:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('inventory: missing after publish');
|
||||
} else {
|
||||
value = sealFacet(value, hit.length);
|
||||
tags.push('inventory.publ');
|
||||
}
|
||||
}
|
||||
|
||||
// 3. warehouse
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('warehouse:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('warehouse: missing after publish');
|
||||
} else {
|
||||
value = rankFacet(value, hit.length);
|
||||
tags.push('warehouse.publ');
|
||||
}
|
||||
}
|
||||
|
||||
// 4. carrier
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('carrier:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('carrier: missing after publish');
|
||||
} else {
|
||||
value = sealFacet(value, hit.length);
|
||||
tags.push('carrier.publ');
|
||||
}
|
||||
}
|
||||
|
||||
// 5. customs
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('customs:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('customs: missing after publish');
|
||||
} else {
|
||||
value = rankFacet(value, hit.length);
|
||||
tags.push('customs.publ');
|
||||
}
|
||||
}
|
||||
|
||||
// 6. tariff
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('tariff:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('tariff: missing after publish');
|
||||
} else {
|
||||
value = sealFacet(value, hit.length);
|
||||
tags.push('tariff.publ');
|
||||
}
|
||||
}
|
||||
|
||||
// 7. sensor
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('sensor:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('sensor: missing after publish');
|
||||
} else {
|
||||
value = rankFacet(value, hit.length);
|
||||
tags.push('sensor.publ');
|
||||
}
|
||||
}
|
||||
|
||||
// 8. firmware
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('firmware:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('firmware: missing after publish');
|
||||
} else {
|
||||
value = sealFacet(value, hit.length);
|
||||
tags.push('firmware.publ');
|
||||
}
|
||||
}
|
||||
|
||||
// 9. telemetry
|
||||
{
|
||||
const hit = tags.find((t) => t.startsWith('telemetry:'));
|
||||
if (hit === undefined) {
|
||||
if (options.strict) warnings.push('telemetry: missing after publish');
|
||||
} else {
|
||||
value = rankFacet(value, hit.length);
|
||||
tags.push('telemetry.publ');
|
||||
}
|
||||
}
|
||||
|
||||
out.push({ ...record, value, tags: tags.slice(0, options.maxTags), warnings });
|
||||
}
|
||||
writeBatch('publishRecords', out);
|
||||
return out;
|
||||
}
|
||||
|
||||
/** rankFacet — a small deterministic helper. */
|
||||
export function rankFacet(base: number, width: number): number {
|
||||
const scaled = base + width * 3 - (width % 7);
|
||||
return scaled < 0 ? 0 : scaled;
|
||||
}
|
||||
|
||||
/** sealFacet — a small deterministic helper. */
|
||||
export function sealFacet(base: number, width: number): number {
|
||||
const scaled = base + width * 3 - (width % 7);
|
||||
return scaled < 0 ? 0 : scaled;
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
import type { PipelineRecord } from './types';
|
||||
|
||||
const sink = new Map<string, PipelineRecord[]>();
|
||||
|
||||
/** Hand a finished batch to the downstream sink. */
|
||||
export function writeBatch(batchId: string, records: PipelineRecord[]): void {
|
||||
sink.set(batchId, records);
|
||||
}
|
||||
|
||||
/** Read a batch back out of the sink. */
|
||||
export function readBatch(batchId: string): PipelineRecord[] {
|
||||
return sink.get(batchId) ?? [];
|
||||
}
|
||||
|
||||
/** Forget a batch. */
|
||||
export function dropBatch(batchId: string): void {
|
||||
sink.delete(batchId);
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
/** One raw record as it arrives from the upstream feed. */
|
||||
export interface RawRecord {
|
||||
id: string;
|
||||
source: string;
|
||||
payload: Record<string, string | number | null>;
|
||||
receivedAt: number;
|
||||
}
|
||||
|
||||
/** A record after the pipeline has cleaned and annotated it. */
|
||||
export interface PipelineRecord {
|
||||
id: string;
|
||||
source: string;
|
||||
kind: string;
|
||||
value: number;
|
||||
tags: string[];
|
||||
warnings: string[];
|
||||
}
|
||||
|
||||
/** Per-run knobs shared by every pipeline stage. */
|
||||
export interface PipelineOptions {
|
||||
strict: boolean;
|
||||
dropEmpty: boolean;
|
||||
defaultKind: string;
|
||||
maxTags: number;
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
{
|
||||
"name": "factory-closure-ts",
|
||||
"version": "0.0.0",
|
||||
"private": true
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
import { createDashboardStore } from './stores/dashboard-store';
|
||||
import { createAlertsStore } from './stores/alerts-store';
|
||||
import { mountPanel } from './ui/panel';
|
||||
import { parseFilterText } from './services/filter-parser';
|
||||
import { refreshMetricCache } from './services/metric-service';
|
||||
import type { StoreDeps } from './stores/types';
|
||||
|
||||
/** Wire a dashboard: build both stores, mount the panel, boot it. */
|
||||
export async function startDashboard(deps: StoreDeps, baseUrl: string, dashboardId: string) {
|
||||
const store = createDashboardStore(deps, baseUrl);
|
||||
const alerts = createAlertsStore(deps, baseUrl);
|
||||
const panel = mountPanel(store, dashboardId);
|
||||
await panel.boot();
|
||||
await alerts.refreshAlerts(dashboardId);
|
||||
return { store, alerts, panel };
|
||||
}
|
||||
|
||||
/** Apply the filter bar's text to the dashboard store. */
|
||||
export function searchDashboard(store: ReturnType<typeof createDashboardStore>, text: string) {
|
||||
return store.applyFilter(parseFilterText(text));
|
||||
}
|
||||
|
||||
export { refreshMetricCache };
|
||||
@@ -0,0 +1,25 @@
|
||||
/** Minimal fetch helpers the dashboard store depends on. */
|
||||
|
||||
export interface RequestOptions {
|
||||
retries: number;
|
||||
timeoutMs: number;
|
||||
}
|
||||
|
||||
export const defaultRequestOptions: RequestOptions = { retries: 2, timeoutMs: 5_000 };
|
||||
|
||||
/** Build a query string from a plain record, skipping empty values. */
|
||||
export function toQueryString(params: Record<string, string | number | undefined>): string {
|
||||
const parts: string[] = [];
|
||||
for (const [key, value] of Object.entries(params)) {
|
||||
if (value === undefined || value === '') continue;
|
||||
parts.push(`${encodeURIComponent(key)}=${encodeURIComponent(String(value))}`);
|
||||
}
|
||||
return parts.length > 0 ? `?${parts.join('&')}` : '';
|
||||
}
|
||||
|
||||
/** Join a base path and a resource path without doubling the separator. */
|
||||
export function joinPath(base: string, resource: string): string {
|
||||
if (base.endsWith('/') && resource.startsWith('/')) return base + resource.slice(1);
|
||||
if (!base.endsWith('/') && !resource.startsWith('/')) return `${base}/${resource}`;
|
||||
return base + resource;
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
import type { MetricSample } from '../stores/types';
|
||||
|
||||
/** Statistics helpers shared by the store and the panel. */
|
||||
|
||||
export function meanOf(samples: readonly MetricSample[]): number {
|
||||
if (samples.length === 0) return 0;
|
||||
let total = 0;
|
||||
for (const sample of samples) total += sample.value;
|
||||
return total / samples.length;
|
||||
}
|
||||
|
||||
export function medianOf(samples: readonly MetricSample[]): number {
|
||||
if (samples.length === 0) return 0;
|
||||
const values = samples.map((s) => s.value).sort((a, b) => a - b);
|
||||
const mid = Math.floor(values.length / 2);
|
||||
return values.length % 2 === 0 ? (values[mid - 1]! + values[mid]!) / 2 : values[mid]!;
|
||||
}
|
||||
|
||||
export function rateOfChange(samples: readonly MetricSample[]): number {
|
||||
if (samples.length < 2) return 0;
|
||||
const ordered = samples.slice().sort((a, b) => a.at - b.at);
|
||||
const first = ordered[0]!;
|
||||
const last = ordered[ordered.length - 1]!;
|
||||
const elapsed = last.at - first.at;
|
||||
return elapsed > 0 ? (last.value - first.value) / elapsed : 0;
|
||||
}
|
||||
|
||||
export function bucketByHour(samples: readonly MetricSample[]): Map<number, MetricSample[]> {
|
||||
const buckets = new Map<number, MetricSample[]>();
|
||||
for (const sample of samples) {
|
||||
const hour = Math.floor(sample.at / 3_600_000);
|
||||
const bucket = buckets.get(hour);
|
||||
if (bucket) bucket.push(sample);
|
||||
else buckets.set(hour, [sample]);
|
||||
}
|
||||
return buckets;
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
import type { FilterSpec } from '../stores/types';
|
||||
|
||||
/** Parse the dashboard's filter bar text into filter specs. */
|
||||
|
||||
const OPERATORS: Record<string, FilterSpec['op']> = {
|
||||
':': 'eq',
|
||||
'~': 'contains',
|
||||
'>': 'gt',
|
||||
'<': 'lt',
|
||||
};
|
||||
|
||||
/** `title~sales kind:chart column>3` → three specs. */
|
||||
export function parseFilterText(text: string): FilterSpec[] {
|
||||
const specs: FilterSpec[] = [];
|
||||
for (const token of tokenize(text)) {
|
||||
const spec = parseToken(token);
|
||||
if (spec) specs.push(spec);
|
||||
}
|
||||
return specs;
|
||||
}
|
||||
|
||||
/** Split on whitespace, honouring double-quoted values. */
|
||||
export function tokenize(text: string): string[] {
|
||||
const tokens: string[] = [];
|
||||
let current = '';
|
||||
let quoted = false;
|
||||
for (const ch of text) {
|
||||
if (ch === '"') { quoted = !quoted; continue; }
|
||||
if (!quoted && /\s/.test(ch)) {
|
||||
if (current.length > 0) { tokens.push(current); current = ''; }
|
||||
continue;
|
||||
}
|
||||
current += ch;
|
||||
}
|
||||
if (current.length > 0) tokens.push(current);
|
||||
return tokens;
|
||||
}
|
||||
|
||||
/** One `field<op>value` token, or null when it does not parse. */
|
||||
export function parseToken(token: string): FilterSpec | null {
|
||||
for (const [symbol, op] of Object.entries(OPERATORS)) {
|
||||
const at = token.indexOf(symbol);
|
||||
if (at <= 0) continue;
|
||||
const field = token.slice(0, at).trim();
|
||||
const value = token.slice(at + symbol.length).trim();
|
||||
if (field.length === 0 || value.length === 0) return null;
|
||||
return { field, op, value };
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/** Render specs back to filter-bar text — the round trip the URL uses. */
|
||||
export function formatFilterText(specs: readonly FilterSpec[]): string {
|
||||
const symbolFor = (op: FilterSpec['op']): string =>
|
||||
Object.entries(OPERATORS).find(([, candidate]) => candidate === op)?.[0] ?? ':';
|
||||
return specs
|
||||
.map((spec) => {
|
||||
const value = /\s/.test(spec.value) ? `"${spec.value}"` : spec.value;
|
||||
return `${spec.field}${symbolFor(spec.op)}${value}`;
|
||||
})
|
||||
.join(' ');
|
||||
}
|
||||
@@ -0,0 +1,101 @@
|
||||
import type { FilterSpec, MetricSample, Widget } from '../stores/types';
|
||||
import { bucketByHour, meanOf, rateOfChange } from '../lib/metrics';
|
||||
|
||||
/**
|
||||
* Stateless metric helpers — the server-shaped half of the same domain. These
|
||||
* are ordinary top-level functions, not closures, so they are the control the
|
||||
* factory-closure file is measured against.
|
||||
*/
|
||||
|
||||
const STALE_AFTER_MS = 15 * 60 * 1000;
|
||||
|
||||
/** Refresh a cached metric map in place, returning the widgets that changed. */
|
||||
export function refreshMetricCache(
|
||||
cache: Map<string, MetricSample[]>,
|
||||
incoming: readonly MetricSample[],
|
||||
now: number,
|
||||
): string[] {
|
||||
const touched = new Set<string>();
|
||||
for (const sample of incoming) {
|
||||
if (typeof sample.value !== 'number' || Number.isNaN(sample.value)) continue;
|
||||
const bucket = cache.get(sample.widgetId);
|
||||
if (bucket) bucket.push(sample);
|
||||
else cache.set(sample.widgetId, [sample]);
|
||||
touched.add(sample.widgetId);
|
||||
}
|
||||
for (const [widgetId, bucket] of cache) {
|
||||
const fresh = bucket.filter((s) => now - s.at <= STALE_AFTER_MS);
|
||||
if (fresh.length !== bucket.length) {
|
||||
cache.set(widgetId, fresh);
|
||||
touched.add(widgetId);
|
||||
}
|
||||
}
|
||||
return [...touched].sort();
|
||||
}
|
||||
|
||||
/** Apply a filter spec set to raw samples rather than to widgets. */
|
||||
export function filterMetrics(
|
||||
samples: readonly MetricSample[],
|
||||
specs: readonly FilterSpec[],
|
||||
): MetricSample[] {
|
||||
if (specs.length === 0) return samples.slice();
|
||||
return samples.filter((sample) => specs.every((spec) => {
|
||||
const field = spec.field === 'unit'
|
||||
? sample.unit
|
||||
: spec.field === 'widget'
|
||||
? sample.widgetId
|
||||
: String(sample.value);
|
||||
switch (spec.op) {
|
||||
case 'eq': return field === spec.value;
|
||||
case 'contains': return field.includes(spec.value);
|
||||
case 'gt': return Number(field) > Number(spec.value);
|
||||
case 'lt': return Number(field) < Number(spec.value);
|
||||
default: return false;
|
||||
}
|
||||
}));
|
||||
}
|
||||
|
||||
/** Per-widget rollup used by the server-rendered summary card. */
|
||||
export function rollupByWidget(
|
||||
samples: readonly MetricSample[],
|
||||
widgets: readonly Widget[],
|
||||
): Array<{ widgetId: string; title: string; mean: number; slope: number; hours: number }> {
|
||||
const titles = new Map(widgets.map((w) => [w.id, w.title]));
|
||||
const grouped = new Map<string, MetricSample[]>();
|
||||
for (const sample of samples) {
|
||||
const bucket = grouped.get(sample.widgetId);
|
||||
if (bucket) bucket.push(sample);
|
||||
else grouped.set(sample.widgetId, [sample]);
|
||||
}
|
||||
|
||||
const out: Array<{ widgetId: string; title: string; mean: number; slope: number; hours: number }> = [];
|
||||
for (const [widgetId, bucket] of grouped) {
|
||||
out.push({
|
||||
widgetId,
|
||||
title: titles.get(widgetId) ?? '(unknown)',
|
||||
mean: meanOf(bucket),
|
||||
slope: rateOfChange(bucket),
|
||||
hours: bucketByHour(bucket).size,
|
||||
});
|
||||
}
|
||||
out.sort((a, b) => b.mean - a.mean);
|
||||
return out;
|
||||
}
|
||||
|
||||
/** Which widgets have not reported inside the staleness window. */
|
||||
export function staleWidgets(
|
||||
samples: readonly MetricSample[],
|
||||
widgets: readonly Widget[],
|
||||
now: number,
|
||||
): string[] {
|
||||
const newest = new Map<string, number>();
|
||||
for (const sample of samples) {
|
||||
const seen = newest.get(sample.widgetId) ?? 0;
|
||||
if (sample.at > seen) newest.set(sample.widgetId, sample.at);
|
||||
}
|
||||
return widgets
|
||||
.filter((w) => !w.hidden)
|
||||
.filter((w) => now - (newest.get(w.id) ?? 0) > STALE_AFTER_MS)
|
||||
.map((w) => w.id)
|
||||
.sort();
|
||||
}
|
||||
@@ -0,0 +1,140 @@
|
||||
import type { FilterSpec, StoreDeps } from './types';
|
||||
import { joinPath, toQueryString } from '../lib/http';
|
||||
|
||||
const ALERT_ENDPOINT = '/api/dashboard/alerts';
|
||||
|
||||
export interface Alert {
|
||||
id: string;
|
||||
widgetId: string;
|
||||
severity: 'info' | 'warn' | 'critical';
|
||||
message: string;
|
||||
raisedAt: number;
|
||||
acknowledgedAt: number | null;
|
||||
}
|
||||
|
||||
/**
|
||||
* The alerts store — the dashboard's second factory closure. Same shape as the
|
||||
* metric store: every operation is a closure over private state.
|
||||
*/
|
||||
export function createAlertsStore(deps: StoreDeps, baseUrl: string) {
|
||||
let alerts: Alert[] = [];
|
||||
let filters: FilterSpec[] = [];
|
||||
let mutedWidgets = new Set<string>();
|
||||
let lastRefreshedAt = 0;
|
||||
|
||||
/** Pull the current alert set and merge acknowledgements the user made locally. */
|
||||
async function refreshAlerts(dashboardId: string): Promise<Alert[]> {
|
||||
const url = joinPath(baseUrl, ALERT_ENDPOINT) + toQueryString({ dashboard: dashboardId });
|
||||
let payload: unknown;
|
||||
try {
|
||||
payload = await deps.fetchJson(url);
|
||||
} catch (error) {
|
||||
deps.log(`refreshAlerts failed: ${error instanceof Error ? error.message : String(error)}`);
|
||||
return alerts;
|
||||
}
|
||||
if (!Array.isArray(payload)) {
|
||||
deps.log('refreshAlerts got a non-array payload');
|
||||
return alerts;
|
||||
}
|
||||
|
||||
const acknowledged = new Map(
|
||||
alerts.filter((a) => a.acknowledgedAt !== null).map((a) => [a.id, a.acknowledgedAt]),
|
||||
);
|
||||
const merged: Alert[] = [];
|
||||
for (const raw of payload as Alert[]) {
|
||||
if (typeof raw.id !== 'string' || raw.id.length === 0) continue;
|
||||
merged.push({
|
||||
...raw,
|
||||
acknowledgedAt: acknowledged.get(raw.id) ?? raw.acknowledgedAt ?? null,
|
||||
});
|
||||
}
|
||||
merged.sort((a, b) => b.raisedAt - a.raisedAt);
|
||||
alerts = merged;
|
||||
lastRefreshedAt = deps.now();
|
||||
return alerts;
|
||||
}
|
||||
|
||||
/** Filter the alert list the same way the metric store filters widgets. */
|
||||
function applyAlertFilter(specs: readonly FilterSpec[]): Alert[] {
|
||||
filters = specs.slice();
|
||||
if (filters.length === 0) return alerts;
|
||||
|
||||
const fieldOf = (alert: Alert, field: string): string => {
|
||||
switch (field) {
|
||||
case 'severity': return alert.severity;
|
||||
case 'widget': return alert.widgetId;
|
||||
case 'message': return alert.message;
|
||||
default: return '';
|
||||
}
|
||||
};
|
||||
|
||||
return alerts.filter((alert) => filters.every((spec) => {
|
||||
const value = fieldOf(alert, spec.field);
|
||||
switch (spec.op) {
|
||||
case 'eq': return value.toLowerCase() === spec.value.toLowerCase();
|
||||
case 'contains': return value.toLowerCase().includes(spec.value.toLowerCase());
|
||||
case 'gt': return value > spec.value;
|
||||
case 'lt': return value < spec.value;
|
||||
default: return false;
|
||||
}
|
||||
}));
|
||||
}
|
||||
|
||||
/** Mark an alert acknowledged locally; the next refresh preserves it. */
|
||||
function acknowledge(alertId: string): boolean {
|
||||
const target = alerts.find((a) => a.id === alertId);
|
||||
if (!target || target.acknowledgedAt !== null) return false;
|
||||
target.acknowledgedAt = deps.now();
|
||||
deps.log(`acknowledged ${alertId}`);
|
||||
return true;
|
||||
}
|
||||
|
||||
/** Silence a widget's alerts without dropping them from the buffer. */
|
||||
function muteWidget(widgetId: string): void {
|
||||
mutedWidgets.add(widgetId);
|
||||
deps.log(`muted ${widgetId} (${mutedWidgets.size} muted)`);
|
||||
}
|
||||
|
||||
function unmuteWidget(widgetId: string): boolean {
|
||||
return mutedWidgets.delete(widgetId);
|
||||
}
|
||||
|
||||
/** The alerts the dashboard should actually show right now. */
|
||||
function visibleAlerts(): Alert[] {
|
||||
return applyAlertFilter(filters)
|
||||
.filter((a) => !mutedWidgets.has(a.widgetId))
|
||||
.filter((a) => a.acknowledgedAt === null);
|
||||
}
|
||||
|
||||
/** Counts per severity, for the badge on the alerts tab. */
|
||||
function countBySeverity(): Record<Alert['severity'], number> {
|
||||
const counts: Record<Alert['severity'], number> = { info: 0, warn: 0, critical: 0 };
|
||||
for (const alert of visibleAlerts()) counts[alert.severity] += 1;
|
||||
return counts;
|
||||
}
|
||||
|
||||
function reset(): void {
|
||||
alerts = [];
|
||||
filters = [];
|
||||
mutedWidgets = new Set();
|
||||
lastRefreshedAt = 0;
|
||||
}
|
||||
|
||||
function snapshot() {
|
||||
return { alerts: visibleAlerts(), counts: countBySeverity(), lastRefreshedAt };
|
||||
}
|
||||
|
||||
return {
|
||||
refreshAlerts,
|
||||
applyAlertFilter,
|
||||
acknowledge,
|
||||
muteWidget,
|
||||
unmuteWidget,
|
||||
visibleAlerts,
|
||||
countBySeverity,
|
||||
reset,
|
||||
snapshot,
|
||||
};
|
||||
}
|
||||
|
||||
export type AlertsStore = ReturnType<typeof createAlertsStore>;
|
||||
@@ -0,0 +1,384 @@
|
||||
import type { FilterSpec, MetricSample, StoreDeps, Widget } from './types';
|
||||
import { defaultRequestOptions, joinPath, toQueryString } from '../lib/http';
|
||||
|
||||
const WIDGET_ENDPOINT = '/api/dashboard/widgets';
|
||||
const METRIC_ENDPOINT = '/api/dashboard/metrics';
|
||||
const SAMPLE_RETENTION_MS = 6 * 60 * 60 * 1000;
|
||||
const MAX_SAMPLES_PER_WIDGET = 720;
|
||||
const COLUMN_COUNT = 12;
|
||||
|
||||
/**
|
||||
* The dashboard store: one factory closure holding every operation the
|
||||
* dashboard performs. Callers get an object of closures; nothing inside is
|
||||
* exported on its own.
|
||||
*/
|
||||
export function createDashboardStore(deps: StoreDeps, baseUrl: string) {
|
||||
let widgets: Widget[] = [];
|
||||
let samples: MetricSample[] = [];
|
||||
let activeFilters: FilterSpec[] = [];
|
||||
let lastSyncedAt = 0;
|
||||
let loading = false;
|
||||
let lastError: string | null = null;
|
||||
const listeners = new Set<(snapshot: ReturnType<typeof snapshot>) => void>();
|
||||
|
||||
function snapshot() {
|
||||
return {
|
||||
widgets: widgets.filter((w) => !w.hidden),
|
||||
sampleCount: samples.length,
|
||||
filters: activeFilters.slice(),
|
||||
lastSyncedAt,
|
||||
loading,
|
||||
lastError,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetch the widget set for the current user and merge it into local state,
|
||||
* preserving any layout the user has moved since the last sync.
|
||||
*/
|
||||
async function loadWidgets(dashboardId: string, includeHidden = false): Promise<Widget[]> {
|
||||
loading = true;
|
||||
lastError = null;
|
||||
const url = joinPath(baseUrl, WIDGET_ENDPOINT) + toQueryString({
|
||||
dashboard: dashboardId,
|
||||
hidden: includeHidden ? '1' : undefined,
|
||||
});
|
||||
|
||||
let attempt = 0;
|
||||
let payload: unknown = null;
|
||||
while (attempt <= defaultRequestOptions.retries) {
|
||||
try {
|
||||
payload = await deps.fetchJson(url);
|
||||
break;
|
||||
} catch (error) {
|
||||
attempt += 1;
|
||||
if (attempt > defaultRequestOptions.retries) {
|
||||
lastError = error instanceof Error ? error.message : String(error);
|
||||
loading = false;
|
||||
deps.log(`loadWidgets failed after ${attempt} attempts: ${lastError}`);
|
||||
notify();
|
||||
return widgets;
|
||||
}
|
||||
deps.log(`loadWidgets retry ${attempt} for ${dashboardId}`);
|
||||
}
|
||||
}
|
||||
|
||||
const incoming = Array.isArray(payload) ? (payload as Widget[]) : [];
|
||||
const byId = new Map(widgets.map((w) => [w.id, w]));
|
||||
const merged: Widget[] = [];
|
||||
for (const next of incoming) {
|
||||
const existing = byId.get(next.id);
|
||||
if (!existing) {
|
||||
merged.push({ ...next });
|
||||
continue;
|
||||
}
|
||||
// Server owns identity and content; the client owns placement.
|
||||
merged.push({
|
||||
...next,
|
||||
column: existing.column,
|
||||
row: existing.row,
|
||||
span: existing.span,
|
||||
hidden: existing.hidden,
|
||||
});
|
||||
byId.delete(next.id);
|
||||
}
|
||||
for (const orphan of byId.values()) {
|
||||
deps.log(`widget ${orphan.id} no longer exists on the server`);
|
||||
}
|
||||
|
||||
widgets = merged;
|
||||
lastSyncedAt = deps.now();
|
||||
loading = false;
|
||||
notify();
|
||||
return widgets;
|
||||
}
|
||||
|
||||
/**
|
||||
* Pull fresh metric samples for every visible widget, append them to the
|
||||
* rolling buffer, and drop anything past the retention window.
|
||||
*/
|
||||
async function refreshMetrics(windowMs = SAMPLE_RETENTION_MS): Promise<MetricSample[]> {
|
||||
if (widgets.length === 0) {
|
||||
deps.log('refreshMetrics called with no widgets loaded');
|
||||
return samples;
|
||||
}
|
||||
loading = true;
|
||||
const visible = widgets.filter((w) => !w.hidden);
|
||||
const collected: MetricSample[] = [];
|
||||
|
||||
for (const widget of visible) {
|
||||
const url = joinPath(baseUrl, METRIC_ENDPOINT) + toQueryString({
|
||||
widget: widget.id,
|
||||
since: deps.now() - windowMs,
|
||||
});
|
||||
let payload: unknown;
|
||||
try {
|
||||
payload = await deps.fetchJson(url);
|
||||
} catch (error) {
|
||||
lastError = error instanceof Error ? error.message : String(error);
|
||||
deps.log(`refreshMetrics failed for ${widget.id}: ${lastError}`);
|
||||
continue;
|
||||
}
|
||||
if (!Array.isArray(payload)) {
|
||||
deps.log(`refreshMetrics got a non-array payload for ${widget.id}`);
|
||||
continue;
|
||||
}
|
||||
for (const raw of payload as MetricSample[]) {
|
||||
if (typeof raw.value !== 'number' || Number.isNaN(raw.value)) continue;
|
||||
if (typeof raw.at !== 'number' || raw.at <= 0) continue;
|
||||
collected.push({
|
||||
widgetId: widget.id,
|
||||
at: raw.at,
|
||||
value: raw.value,
|
||||
unit: raw.unit ?? 'count',
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const cutoff = deps.now() - windowMs;
|
||||
const kept = samples.filter((s) => s.at >= cutoff);
|
||||
samples = kept.concat(collected);
|
||||
pruneSamples(MAX_SAMPLES_PER_WIDGET);
|
||||
lastSyncedAt = deps.now();
|
||||
loading = false;
|
||||
notify();
|
||||
return samples;
|
||||
}
|
||||
|
||||
/**
|
||||
* Replace the active filter set and recompute which widgets stay visible.
|
||||
* A widget survives when every filter matches one of its fields.
|
||||
*/
|
||||
function applyFilter(specs: readonly FilterSpec[]): Widget[] {
|
||||
activeFilters = specs.slice();
|
||||
if (activeFilters.length === 0) {
|
||||
widgets = widgets.map((w) => ({ ...w, hidden: false }));
|
||||
notify();
|
||||
return widgets;
|
||||
}
|
||||
|
||||
const matches = (widget: Widget, spec: FilterSpec): boolean => {
|
||||
const field = spec.field === 'title'
|
||||
? widget.title
|
||||
: spec.field === 'kind'
|
||||
? widget.kind
|
||||
: spec.field === 'column'
|
||||
? String(widget.column)
|
||||
: '';
|
||||
switch (spec.op) {
|
||||
case 'eq':
|
||||
return field.toLowerCase() === spec.value.toLowerCase();
|
||||
case 'contains':
|
||||
return field.toLowerCase().includes(spec.value.toLowerCase());
|
||||
case 'gt':
|
||||
return Number(field) > Number(spec.value);
|
||||
case 'lt':
|
||||
return Number(field) < Number(spec.value);
|
||||
default:
|
||||
return false;
|
||||
}
|
||||
};
|
||||
|
||||
let hiddenCount = 0;
|
||||
widgets = widgets.map((widget) => {
|
||||
const visible = activeFilters.every((spec) => matches(widget, spec));
|
||||
if (!visible) hiddenCount += 1;
|
||||
return { ...widget, hidden: !visible };
|
||||
});
|
||||
deps.log(`applyFilter hid ${hiddenCount} of ${widgets.length} widgets`);
|
||||
notify();
|
||||
return widgets;
|
||||
}
|
||||
|
||||
/**
|
||||
* Render the current sample buffer as CSV, one row per sample, ordered by
|
||||
* widget then timestamp so a diff between two exports stays readable.
|
||||
*/
|
||||
function exportCsv(separator = ','): string {
|
||||
const header = ['widget', 'title', 'at', 'value', 'unit'].join(separator);
|
||||
if (samples.length === 0) return header;
|
||||
|
||||
const titles = new Map(widgets.map((w) => [w.id, w.title]));
|
||||
const ordered = samples.slice().sort((a, b) => {
|
||||
if (a.widgetId !== b.widgetId) return a.widgetId < b.widgetId ? -1 : 1;
|
||||
return a.at - b.at;
|
||||
});
|
||||
|
||||
const escape = (value: string): string => {
|
||||
if (!value.includes(separator) && !value.includes('"') && !value.includes('\n')) return value;
|
||||
return `"${value.replace(/"/g, '""')}"`;
|
||||
};
|
||||
|
||||
const rows = ordered.map((sample) => [
|
||||
escape(sample.widgetId),
|
||||
escape(titles.get(sample.widgetId) ?? '(unknown)'),
|
||||
String(sample.at),
|
||||
String(sample.value),
|
||||
escape(sample.unit),
|
||||
].join(separator));
|
||||
|
||||
return [header, ...rows].join('\n');
|
||||
}
|
||||
|
||||
/**
|
||||
* Pack widgets back into a dense grid after a move or a hide, so the layout
|
||||
* never leaves a hole a user has to scroll past.
|
||||
*/
|
||||
function reconcileLayout(columnCount = COLUMN_COUNT): Widget[] {
|
||||
const visible = widgets.filter((w) => !w.hidden);
|
||||
const hidden = widgets.filter((w) => w.hidden);
|
||||
|
||||
const ordered = visible.slice().sort((a, b) => {
|
||||
if (a.row !== b.row) return a.row - b.row;
|
||||
return a.column - b.column;
|
||||
});
|
||||
|
||||
const rowWidth = new Map<number, number>();
|
||||
const placed: Widget[] = [];
|
||||
for (const widget of ordered) {
|
||||
const span = Math.max(1, Math.min(widget.span, columnCount));
|
||||
let row = 0;
|
||||
let column = 0;
|
||||
for (;;) {
|
||||
const used = rowWidth.get(row) ?? 0;
|
||||
if (used + span <= columnCount) {
|
||||
column = used;
|
||||
rowWidth.set(row, used + span);
|
||||
break;
|
||||
}
|
||||
row += 1;
|
||||
}
|
||||
placed.push({ ...widget, row, column, span });
|
||||
}
|
||||
|
||||
let trailing = placed.length > 0 ? Math.max(...placed.map((w) => w.row)) + 1 : 0;
|
||||
for (const widget of hidden) {
|
||||
placed.push({ ...widget, row: trailing, column: 0 });
|
||||
trailing += 1;
|
||||
}
|
||||
|
||||
widgets = placed;
|
||||
notify();
|
||||
return widgets;
|
||||
}
|
||||
|
||||
/**
|
||||
* Cap the rolling buffer per widget, keeping the newest samples. Called after
|
||||
* every refresh so memory stays bounded on a long-lived dashboard.
|
||||
*/
|
||||
function pruneSamples(perWidget = MAX_SAMPLES_PER_WIDGET): number {
|
||||
if (samples.length === 0) return 0;
|
||||
const grouped = new Map<string, MetricSample[]>();
|
||||
for (const sample of samples) {
|
||||
const bucket = grouped.get(sample.widgetId);
|
||||
if (bucket) bucket.push(sample);
|
||||
else grouped.set(sample.widgetId, [sample]);
|
||||
}
|
||||
|
||||
let dropped = 0;
|
||||
const kept: MetricSample[] = [];
|
||||
for (const [, bucket] of grouped) {
|
||||
bucket.sort((a, b) => a.at - b.at);
|
||||
if (bucket.length > perWidget) {
|
||||
dropped += bucket.length - perWidget;
|
||||
kept.push(...bucket.slice(bucket.length - perWidget));
|
||||
} else {
|
||||
kept.push(...bucket);
|
||||
}
|
||||
}
|
||||
|
||||
kept.sort((a, b) => a.at - b.at);
|
||||
samples = kept;
|
||||
if (dropped > 0) deps.log(`pruneSamples dropped ${dropped} samples`);
|
||||
return dropped;
|
||||
}
|
||||
|
||||
/**
|
||||
* Reduce the buffer to one aggregate per widget — the numbers the summary
|
||||
* strip at the top of the dashboard renders.
|
||||
*/
|
||||
function summarize(): Array<{ widgetId: string; title: string; min: number; max: number; mean: number; count: number }> {
|
||||
const titles = new Map(widgets.map((w) => [w.id, w.title]));
|
||||
const grouped = new Map<string, MetricSample[]>();
|
||||
for (const sample of samples) {
|
||||
const bucket = grouped.get(sample.widgetId);
|
||||
if (bucket) bucket.push(sample);
|
||||
else grouped.set(sample.widgetId, [sample]);
|
||||
}
|
||||
|
||||
const out: Array<{ widgetId: string; title: string; min: number; max: number; mean: number; count: number }> = [];
|
||||
for (const [widgetId, bucket] of grouped) {
|
||||
let min = Number.POSITIVE_INFINITY;
|
||||
let max = Number.NEGATIVE_INFINITY;
|
||||
let total = 0;
|
||||
for (const sample of bucket) {
|
||||
if (sample.value < min) min = sample.value;
|
||||
if (sample.value > max) max = sample.value;
|
||||
total += sample.value;
|
||||
}
|
||||
out.push({
|
||||
widgetId,
|
||||
title: titles.get(widgetId) ?? '(unknown)',
|
||||
min: bucket.length > 0 ? min : 0,
|
||||
max: bucket.length > 0 ? max : 0,
|
||||
mean: bucket.length > 0 ? total / bucket.length : 0,
|
||||
count: bucket.length,
|
||||
});
|
||||
}
|
||||
|
||||
out.sort((a, b) => b.count - a.count || (a.title < b.title ? -1 : 1));
|
||||
return out;
|
||||
}
|
||||
|
||||
/** Register a listener and get an unsubscribe back. */
|
||||
function subscribe(listener: (snapshot: ReturnType<typeof snapshot>) => void): () => void {
|
||||
listeners.add(listener);
|
||||
listener(snapshot());
|
||||
return () => {
|
||||
listeners.delete(listener);
|
||||
};
|
||||
}
|
||||
|
||||
function notify(): void {
|
||||
const current = snapshot();
|
||||
for (const listener of listeners) {
|
||||
try {
|
||||
listener(current);
|
||||
} catch (error) {
|
||||
deps.log(`dashboard listener threw: ${error instanceof Error ? error.message : String(error)}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Drop every sample and widget — used when the user switches dashboards. */
|
||||
function reset(): void {
|
||||
widgets = [];
|
||||
samples = [];
|
||||
activeFilters = [];
|
||||
lastSyncedAt = 0;
|
||||
lastError = null;
|
||||
loading = false;
|
||||
notify();
|
||||
}
|
||||
|
||||
return {
|
||||
loadWidgets,
|
||||
refreshMetrics,
|
||||
applyFilter,
|
||||
exportCsv,
|
||||
reconcileLayout,
|
||||
pruneSamples,
|
||||
summarize,
|
||||
subscribe,
|
||||
reset,
|
||||
snapshot,
|
||||
};
|
||||
}
|
||||
|
||||
export type DashboardStore = ReturnType<typeof createDashboardStore>;
|
||||
|
||||
/** One-line description of a store's state, for the debug panel. */
|
||||
export function describeStore(store: DashboardStore): string {
|
||||
const state = store.snapshot();
|
||||
return `${state.widgets.length} widgets · ${state.sampleCount} samples · synced ${state.lastSyncedAt}`;
|
||||
}
|
||||
@@ -0,0 +1,148 @@
|
||||
import type { StoreDeps } from './types';
|
||||
import { joinPath, toQueryString } from '../lib/http';
|
||||
|
||||
/**
|
||||
* The session store — a factory closure and NOTHING else at file scope. No
|
||||
* companion type alias, no tail helper, no exported constants: every other
|
||||
* symbol in this file lives inside the closure. That shape matters, because it
|
||||
* is the one where the enclosing range is the only top-importance symbol the
|
||||
* file can offer a query.
|
||||
*/
|
||||
export function createSessionStore(deps: StoreDeps, baseUrl: string) {
|
||||
const SESSION_ENDPOINT = '/api/session';
|
||||
const REFRESH_SKEW_MS = 30_000;
|
||||
|
||||
let token: string | null = null;
|
||||
let expiresAt = 0;
|
||||
let profile: { id: string; email: string; roles: string[] } | null = null;
|
||||
let refreshing: Promise<string | null> | null = null;
|
||||
const auditLog: Array<{ at: number; event: string }> = [];
|
||||
|
||||
function record(event: string): void {
|
||||
auditLog.push({ at: deps.now(), event });
|
||||
if (auditLog.length > 200) auditLog.splice(0, auditLog.length - 200);
|
||||
}
|
||||
|
||||
/** Exchange credentials for a session token and cache the profile. */
|
||||
async function signIn(email: string, password: string): Promise<boolean> {
|
||||
const url = joinPath(baseUrl, SESSION_ENDPOINT) + toQueryString({ email });
|
||||
let payload: unknown;
|
||||
try {
|
||||
payload = await deps.fetchJson(url);
|
||||
} catch (error) {
|
||||
record(`signIn failed: ${error instanceof Error ? error.message : String(error)}`);
|
||||
return false;
|
||||
}
|
||||
if (typeof payload !== 'object' || payload === null) {
|
||||
record('signIn got a non-object payload');
|
||||
return false;
|
||||
}
|
||||
const body = payload as { token?: string; expiresAt?: number; profile?: typeof profile };
|
||||
if (typeof body.token !== 'string' || body.token.length === 0) {
|
||||
record('signIn payload carried no token');
|
||||
return false;
|
||||
}
|
||||
void password;
|
||||
token = body.token;
|
||||
expiresAt = typeof body.expiresAt === 'number' ? body.expiresAt : deps.now() + 3_600_000;
|
||||
profile = body.profile ?? null;
|
||||
record(`signIn ok for ${email}`);
|
||||
return true;
|
||||
}
|
||||
|
||||
/** Drop every trace of the session, locally and on the server. */
|
||||
async function signOut(): Promise<void> {
|
||||
if (token === null) return;
|
||||
const url = joinPath(baseUrl, SESSION_ENDPOINT) + toQueryString({ action: 'revoke' });
|
||||
try {
|
||||
await deps.fetchJson(url);
|
||||
} catch (error) {
|
||||
record(`signOut revoke failed: ${error instanceof Error ? error.message : String(error)}`);
|
||||
}
|
||||
token = null;
|
||||
expiresAt = 0;
|
||||
profile = null;
|
||||
refreshing = null;
|
||||
record('signOut complete');
|
||||
}
|
||||
|
||||
/**
|
||||
* Renew the token before it expires. Concurrent callers share one in-flight
|
||||
* request so a burst of requests cannot start a refresh storm.
|
||||
*/
|
||||
async function refreshToken(): Promise<string | null> {
|
||||
if (token === null) return null;
|
||||
if (refreshing !== null) return refreshing;
|
||||
|
||||
refreshing = (async () => {
|
||||
const url = joinPath(baseUrl, SESSION_ENDPOINT) + toQueryString({ action: 'refresh' });
|
||||
try {
|
||||
const payload = await deps.fetchJson(url);
|
||||
const body = payload as { token?: string; expiresAt?: number };
|
||||
if (typeof body?.token === 'string' && body.token.length > 0) {
|
||||
token = body.token;
|
||||
expiresAt = typeof body.expiresAt === 'number' ? body.expiresAt : deps.now() + 3_600_000;
|
||||
record('refreshToken renewed the session');
|
||||
return token;
|
||||
}
|
||||
record('refreshToken payload carried no token');
|
||||
return null;
|
||||
} catch (error) {
|
||||
record(`refreshToken failed: ${error instanceof Error ? error.message : String(error)}`);
|
||||
return null;
|
||||
} finally {
|
||||
refreshing = null;
|
||||
}
|
||||
})();
|
||||
|
||||
return refreshing;
|
||||
}
|
||||
|
||||
/** The token to send with a request, renewing it first when it is close to expiry. */
|
||||
async function authorize(): Promise<string | null> {
|
||||
if (token === null) return null;
|
||||
if (deps.now() + REFRESH_SKEW_MS < expiresAt) return token;
|
||||
return refreshToken();
|
||||
}
|
||||
|
||||
/** Does the signed-in user hold every one of these roles? */
|
||||
function hasRoles(...required: string[]): boolean {
|
||||
if (profile === null) return false;
|
||||
const held = new Set(profile.roles);
|
||||
for (const role of required) {
|
||||
if (!held.has(role)) return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
/** Seconds left on the session, floored at zero. */
|
||||
function secondsRemaining(): number {
|
||||
if (token === null) return 0;
|
||||
return Math.max(0, Math.floor((expiresAt - deps.now()) / 1000));
|
||||
}
|
||||
|
||||
/** The last N audit entries, newest first — what the account page renders. */
|
||||
function recentActivity(limit = 20): Array<{ at: number; event: string }> {
|
||||
return auditLog.slice(-limit).reverse();
|
||||
}
|
||||
|
||||
function snapshot() {
|
||||
return {
|
||||
signedIn: token !== null,
|
||||
email: profile?.email ?? null,
|
||||
roles: profile?.roles ?? [],
|
||||
secondsRemaining: secondsRemaining(),
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
signIn,
|
||||
signOut,
|
||||
refreshToken,
|
||||
authorize,
|
||||
hasRoles,
|
||||
secondsRemaining,
|
||||
recentActivity,
|
||||
snapshot,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
export interface Widget {
|
||||
id: string;
|
||||
kind: 'chart' | 'table' | 'stat';
|
||||
title: string;
|
||||
column: number;
|
||||
row: number;
|
||||
span: number;
|
||||
hidden: boolean;
|
||||
}
|
||||
|
||||
export interface MetricSample {
|
||||
widgetId: string;
|
||||
at: number;
|
||||
value: number;
|
||||
unit: string;
|
||||
}
|
||||
|
||||
export interface FilterSpec {
|
||||
field: string;
|
||||
op: 'eq' | 'gt' | 'lt' | 'contains';
|
||||
value: string;
|
||||
}
|
||||
|
||||
export interface StoreDeps {
|
||||
fetchJson: (url: string) => Promise<unknown>;
|
||||
now: () => number;
|
||||
log: (message: string) => void;
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
import type { DashboardStore } from '../stores/dashboard-store';
|
||||
import type { FilterSpec } from '../stores/types';
|
||||
import { medianOf } from '../lib/metrics';
|
||||
|
||||
/** The dashboard panel — the only consumer of the store's closures. */
|
||||
export function mountPanel(store: DashboardStore, dashboardId: string) {
|
||||
let disposed = false;
|
||||
|
||||
const unsubscribe = store.subscribe((state) => {
|
||||
if (disposed) return;
|
||||
render(state.widgets.length, state.sampleCount, state.loading);
|
||||
});
|
||||
|
||||
async function boot(): Promise<void> {
|
||||
await store.loadWidgets(dashboardId);
|
||||
await store.refreshMetrics();
|
||||
store.reconcileLayout();
|
||||
}
|
||||
|
||||
function search(text: string): void {
|
||||
const specs: FilterSpec[] = text.trim().length === 0
|
||||
? []
|
||||
: [{ field: 'title', op: 'contains', value: text.trim() }];
|
||||
store.applyFilter(specs);
|
||||
}
|
||||
|
||||
function download(): string {
|
||||
return store.exportCsv();
|
||||
}
|
||||
|
||||
function render(widgetCount: number, sampleCount: number, loading: boolean): void {
|
||||
void widgetCount;
|
||||
void sampleCount;
|
||||
void loading;
|
||||
}
|
||||
|
||||
function dispose(): void {
|
||||
disposed = true;
|
||||
unsubscribe();
|
||||
}
|
||||
|
||||
return { boot, search, download, dispose, median: medianOf };
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"name": "oversize-member-fixture",
|
||||
"version": "1.0.0",
|
||||
"private": true,
|
||||
"type": "module"
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
import { buildMonthlyReport } from './report/monthly';
|
||||
import { buildWeeklyReport } from './report/weekly';
|
||||
import { buildQuarterlyReport } from './report/quarterly';
|
||||
import { formatReportRows } from './report/format';
|
||||
import type { Ledger, ReportOptions } from './report/types';
|
||||
|
||||
/** Run every report for a ledger and render them. */
|
||||
export function runReports(ledger: Ledger, options: ReportOptions): string {
|
||||
return [
|
||||
formatReportRows(buildMonthlyReport(ledger, options)),
|
||||
formatReportRows(buildWeeklyReport(ledger, options)),
|
||||
formatReportRows(buildQuarterlyReport(ledger, options)),
|
||||
].join('\n\n');
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
import type { ReportRow } from './types';
|
||||
|
||||
/** Format one category total as a report row. */
|
||||
export function formatReportRow(category: string, amountCents: number, currency: string): ReportRow {
|
||||
return {
|
||||
category,
|
||||
amount: formatAmount(amountCents),
|
||||
currency,
|
||||
};
|
||||
}
|
||||
|
||||
/** Render cents as a fixed-point amount. */
|
||||
export function formatAmount(amountCents: number): string {
|
||||
const sign = amountCents < 0 ? '-' : '';
|
||||
const abs = Math.abs(amountCents);
|
||||
return `${sign}${Math.floor(abs / 100)}.${String(abs % 100).padStart(2, '0')}`;
|
||||
}
|
||||
|
||||
/** Render a set of rows as plain text. */
|
||||
export function formatReportRows(rows: ReportRow[]): string {
|
||||
return rows.map((row) => `${row.category}\t${row.amount} ${row.currency}`).join('\n');
|
||||
}
|
||||
@@ -0,0 +1,509 @@
|
||||
import { formatReportRow } from './format';
|
||||
import { persistReport } from './store';
|
||||
import type { Ledger, ReportOptions, ReportRow } from './types';
|
||||
|
||||
/**
|
||||
* Build the monthly report for one ledger.
|
||||
*
|
||||
* Every expense category is accrued in its own block so the finance team can
|
||||
* read the month end-to-end in one place; the shape is deliberately flat.
|
||||
*/
|
||||
export function buildMonthlyReport(ledger: Ledger, options: ReportOptions): ReportRow[] {
|
||||
const rows: ReportRow[] = [];
|
||||
const totals = new Map<string, number>();
|
||||
|
||||
// 1. payroll — accrue the payroll component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'payroll');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('payroll', adjusted, options.currency));
|
||||
totals.set('payroll', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 2. benefits — accrue the benefits component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'benefits');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('benefits', adjusted, options.currency));
|
||||
totals.set('benefits', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 3. travel — accrue the travel component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'travel');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('travel', adjusted, options.currency));
|
||||
totals.set('travel', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 4. equipment — accrue the equipment component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'equipment');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('equipment', adjusted, options.currency));
|
||||
totals.set('equipment', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 5. software — accrue the software component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'software');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('software', adjusted, options.currency));
|
||||
totals.set('software', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 6. contractors — accrue the contractors component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'contractors');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('contractors', adjusted, options.currency));
|
||||
totals.set('contractors', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 7. marketing — accrue the marketing component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'marketing');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('marketing', adjusted, options.currency));
|
||||
totals.set('marketing', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 8. training — accrue the training component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'training');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('training', adjusted, options.currency));
|
||||
totals.set('training', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 9. utilities — accrue the utilities component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'utilities');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('utilities', adjusted, options.currency));
|
||||
totals.set('utilities', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 10. rent — accrue the rent component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'rent');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('rent', adjusted, options.currency));
|
||||
totals.set('rent', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 11. insurance — accrue the insurance component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'insurance');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('insurance', adjusted, options.currency));
|
||||
totals.set('insurance', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 12. legal — accrue the legal component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'legal');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('legal', adjusted, options.currency));
|
||||
totals.set('legal', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 13. shipping — accrue the shipping component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'shipping');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('shipping', adjusted, options.currency));
|
||||
totals.set('shipping', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 14. hosting — accrue the hosting component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'hosting');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('hosting', adjusted, options.currency));
|
||||
totals.set('hosting', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 15. support — accrue the support component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'support');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('support', adjusted, options.currency));
|
||||
totals.set('support', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 16. recruiting — accrue the recruiting component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'recruiting');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('recruiting', adjusted, options.currency));
|
||||
totals.set('recruiting', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 17. licenses — accrue the licenses component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'licenses');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('licenses', adjusted, options.currency));
|
||||
totals.set('licenses', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 18. taxes — accrue the taxes component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'taxes');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('taxes', adjusted, options.currency));
|
||||
totals.set('taxes', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 19. refunds — accrue the refunds component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'refunds');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('refunds', adjusted, options.currency));
|
||||
totals.set('refunds', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 20. discounts — accrue the discounts component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'discounts');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('discounts', adjusted, options.currency));
|
||||
totals.set('discounts', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 21. interest — accrue the interest component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'interest');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('interest', adjusted, options.currency));
|
||||
totals.set('interest', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 22. depreciation — accrue the depreciation component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'depreciation');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('depreciation', adjusted, options.currency));
|
||||
totals.set('depreciation', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 23. maintenance — accrue the maintenance component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'maintenance');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('maintenance', adjusted, options.currency));
|
||||
totals.set('maintenance', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 24. subscriptions — accrue the subscriptions component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'subscriptions');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('subscriptions', adjusted, options.currency));
|
||||
totals.set('subscriptions', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 25. hardware — accrue the hardware component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'hardware');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('hardware', adjusted, options.currency));
|
||||
totals.set('hardware', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 26. catering — accrue the catering component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'catering');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('catering', adjusted, options.currency));
|
||||
totals.set('catering', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 27. conferences — accrue the conferences component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'conferences');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('conferences', adjusted, options.currency));
|
||||
totals.set('conferences', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 28. advertising — accrue the advertising component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'advertising');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('advertising', adjusted, options.currency));
|
||||
totals.set('advertising', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 29. research — accrue the research component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'research');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('research', adjusted, options.currency));
|
||||
totals.set('research', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 30. logistics — accrue the logistics component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'logistics');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('logistics', adjusted, options.currency));
|
||||
totals.set('logistics', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 31. warranty — accrue the warranty component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'warranty');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('warranty', adjusted, options.currency));
|
||||
totals.set('warranty', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 32. penalties — accrue the penalties component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'penalties');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('penalties', adjusted, options.currency));
|
||||
totals.set('penalties', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 33. bonuses — accrue the bonuses component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'bonuses');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('bonuses', adjusted, options.currency));
|
||||
totals.set('bonuses', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 34. commissions — accrue the commissions component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'commissions');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('commissions', adjusted, options.currency));
|
||||
totals.set('commissions', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 35. relocation — accrue the relocation component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'relocation');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('relocation', adjusted, options.currency));
|
||||
totals.set('relocation', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 36. tooling — accrue the tooling component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'tooling');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('tooling', adjusted, options.currency));
|
||||
totals.set('tooling', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 37. audit — accrue the audit component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'audit');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('audit', adjusted, options.currency));
|
||||
totals.set('audit', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 38. compliance — accrue the compliance component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'compliance');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('compliance', adjusted, options.currency));
|
||||
totals.set('compliance', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 39. storage — accrue the storage component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'storage');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('storage', adjusted, options.currency));
|
||||
totals.set('storage', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 40. bandwidth — accrue the bandwidth component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'bandwidth');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('bandwidth', adjusted, options.currency));
|
||||
totals.set('bandwidth', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
const grandTotal = [...totals.values()].reduce((sum, value) => sum + value, 0);
|
||||
rows.push(formatReportRow('total', grandTotal, options.currency));
|
||||
persistReport(ledger.periodId, rows);
|
||||
return rows;
|
||||
}
|
||||
|
||||
/** Header line for a rendered monthly report. */
|
||||
export function monthlyReportHeader(ledger: Ledger, options: ReportOptions): string {
|
||||
return `Monthly report ${ledger.periodId} (${options.currency})`;
|
||||
}
|
||||
|
||||
/** Footer line for a rendered monthly report. */
|
||||
export function monthlyReportFooter(rows: ReportRow[]): string {
|
||||
return `${rows.length} categories reported`;
|
||||
}
|
||||
@@ -0,0 +1,235 @@
|
||||
import { formatReportRow } from './format';
|
||||
import { persistReport } from './store';
|
||||
import type { Ledger, ReportOptions, ReportRow } from './types';
|
||||
|
||||
/** Build the quarterly report for one ledger. */
|
||||
export function buildQuarterlyReport(ledger: Ledger, options: ReportOptions): ReportRow[] {
|
||||
const rows: ReportRow[] = [];
|
||||
const totals = new Map<string, number>();
|
||||
|
||||
// 1. insurance — accrue the insurance component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'insurance');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('insurance', adjusted, options.currency));
|
||||
totals.set('insurance', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 2. legal — accrue the legal component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'legal');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('legal', adjusted, options.currency));
|
||||
totals.set('legal', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 3. shipping — accrue the shipping component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'shipping');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('shipping', adjusted, options.currency));
|
||||
totals.set('shipping', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 4. hosting — accrue the hosting component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'hosting');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('hosting', adjusted, options.currency));
|
||||
totals.set('hosting', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 5. support — accrue the support component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'support');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('support', adjusted, options.currency));
|
||||
totals.set('support', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 6. recruiting — accrue the recruiting component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'recruiting');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('recruiting', adjusted, options.currency));
|
||||
totals.set('recruiting', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 7. licenses — accrue the licenses component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'licenses');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('licenses', adjusted, options.currency));
|
||||
totals.set('licenses', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 8. taxes — accrue the taxes component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'taxes');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('taxes', adjusted, options.currency));
|
||||
totals.set('taxes', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 9. refunds — accrue the refunds component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'refunds');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('refunds', adjusted, options.currency));
|
||||
totals.set('refunds', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 10. discounts — accrue the discounts component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'discounts');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('discounts', adjusted, options.currency));
|
||||
totals.set('discounts', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 11. interest — accrue the interest component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'interest');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('interest', adjusted, options.currency));
|
||||
totals.set('interest', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 12. depreciation — accrue the depreciation component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'depreciation');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('depreciation', adjusted, options.currency));
|
||||
totals.set('depreciation', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 13. maintenance — accrue the maintenance component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'maintenance');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('maintenance', adjusted, options.currency));
|
||||
totals.set('maintenance', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 14. subscriptions — accrue the subscriptions component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'subscriptions');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('subscriptions', adjusted, options.currency));
|
||||
totals.set('subscriptions', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 15. hardware — accrue the hardware component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'hardware');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('hardware', adjusted, options.currency));
|
||||
totals.set('hardware', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 16. catering — accrue the catering component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'catering');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('catering', adjusted, options.currency));
|
||||
totals.set('catering', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 17. conferences — accrue the conferences component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'conferences');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('conferences', adjusted, options.currency));
|
||||
totals.set('conferences', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
// 18. advertising — accrue the advertising component of the month.
|
||||
{
|
||||
const bucket = ledger.entries.filter((entry) => entry.category === 'advertising');
|
||||
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
|
||||
const adjusted = options.includePending ? gross : gross - pending;
|
||||
if (adjusted !== 0 || options.includeEmptyCategories) {
|
||||
rows.push(formatReportRow('advertising', adjusted, options.currency));
|
||||
totals.set('advertising', adjusted);
|
||||
}
|
||||
}
|
||||
|
||||
const grandTotal = [...totals.values()].reduce((sum, value) => sum + value, 0);
|
||||
rows.push(formatReportRow('total', grandTotal, options.currency));
|
||||
persistReport(ledger.periodId, rows);
|
||||
return rows;
|
||||
}
|
||||
|
||||
/** Header line for a rendered quarterly report. */
|
||||
export function buildQuarterlyReportHeader(ledger: Ledger, options: ReportOptions): string {
|
||||
return `quarterly report ${ledger.periodId} (${options.currency})`;
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
import type { ReportRow } from './types';
|
||||
|
||||
const saved = new Map<string, ReportRow[]>();
|
||||
|
||||
/** Persist a built report for a period. */
|
||||
export function persistReport(periodId: string, rows: ReportRow[]): void {
|
||||
saved.set(periodId, rows);
|
||||
}
|
||||
|
||||
/** Read back a persisted report. */
|
||||
export function loadReport(periodId: string): ReportRow[] {
|
||||
return saved.get(periodId) ?? [];
|
||||
}
|
||||
|
||||
/** Drop a persisted report. */
|
||||
export function clearReport(periodId: string): void {
|
||||
saved.delete(periodId);
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
/** One posted ledger entry. */
|
||||
export interface LedgerEntry {
|
||||
id: string;
|
||||
category: string;
|
||||
amountCents: number;
|
||||
pending: boolean;
|
||||
postedAt: string;
|
||||
}
|
||||
|
||||
/** A period's ledger. */
|
||||
export interface Ledger {
|
||||
periodId: string;
|
||||
entries: LedgerEntry[];
|
||||
}
|
||||
|
||||
/** How a report should be built. */
|
||||
export interface ReportOptions {
|
||||
currency: string;
|
||||
includePending: boolean;
|
||||
includeEmptyCategories: boolean;
|
||||
}
|
||||
|
||||
/** One rendered report line. */
|
||||
export interface ReportRow {
|
||||
category: string;
|
||||
amount: string;
|
||||
currency: string;
|
||||
}
|
||||
@@ -0,0 +1,372 @@
|
||||
import { formatReportRow } from './format';
|
||||
import { persistReport } from './store';
|
||||
import type { Ledger, ReportOptions, ReportRow } from './types';
|
||||
|
||||
/** Total the posted entries in one category. */
|
||||
function sumOf(ledger: Ledger, category: string): number {
|
||||
return ledger.entries
|
||||
.filter((entry) => entry.category === category && !entry.pending)
|
||||
.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
}
|
||||
|
||||
/** Total the still-pending entries in one category. */
|
||||
function pendingOf(ledger: Ledger, category: string): number {
|
||||
return ledger.entries
|
||||
.filter((entry) => entry.category === category && entry.pending)
|
||||
.reduce((sum, entry) => sum + entry.amountCents, 0);
|
||||
}
|
||||
|
||||
/** Build the weekly report for one ledger. */
|
||||
export function buildWeeklyReport(ledger: Ledger, options: ReportOptions): ReportRow[] {
|
||||
const rows: ReportRow[] = [];
|
||||
const totals = new Map<string, number>();
|
||||
|
||||
// 1. payroll
|
||||
{
|
||||
const gross = sumOf(ledger, 'payroll');
|
||||
const held = pendingOf(ledger, 'payroll');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('payroll', net, options.currency));
|
||||
totals.set('payroll', net);
|
||||
}
|
||||
}
|
||||
|
||||
// 2. benefits
|
||||
{
|
||||
const gross = sumOf(ledger, 'benefits');
|
||||
const held = pendingOf(ledger, 'benefits');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('benefits', net, options.currency));
|
||||
totals.set('benefits', net);
|
||||
}
|
||||
}
|
||||
|
||||
// 3. travel
|
||||
{
|
||||
const gross = sumOf(ledger, 'travel');
|
||||
const held = pendingOf(ledger, 'travel');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('travel', net, options.currency));
|
||||
totals.set('travel', net);
|
||||
}
|
||||
}
|
||||
|
||||
// 4. equipment
|
||||
{
|
||||
const gross = sumOf(ledger, 'equipment');
|
||||
const held = pendingOf(ledger, 'equipment');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('equipment', net, options.currency));
|
||||
totals.set('equipment', net);
|
||||
}
|
||||
}
|
||||
|
||||
// 5. software
|
||||
{
|
||||
const gross = sumOf(ledger, 'software');
|
||||
const held = pendingOf(ledger, 'software');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('software', net, options.currency));
|
||||
totals.set('software', net);
|
||||
}
|
||||
}
|
||||
|
||||
// 6. contractors
|
||||
{
|
||||
const gross = sumOf(ledger, 'contractors');
|
||||
const held = pendingOf(ledger, 'contractors');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('contractors', net, options.currency));
|
||||
totals.set('contractors', net);
|
||||
}
|
||||
}
|
||||
|
||||
// 7. marketing
|
||||
{
|
||||
const gross = sumOf(ledger, 'marketing');
|
||||
const held = pendingOf(ledger, 'marketing');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('marketing', net, options.currency));
|
||||
totals.set('marketing', net);
|
||||
}
|
||||
}
|
||||
|
||||
// 8. training
|
||||
{
|
||||
const gross = sumOf(ledger, 'training');
|
||||
const held = pendingOf(ledger, 'training');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('training', net, options.currency));
|
||||
totals.set('training', net);
|
||||
}
|
||||
}
|
||||
|
||||
// 9. utilities
|
||||
{
|
||||
const gross = sumOf(ledger, 'utilities');
|
||||
const held = pendingOf(ledger, 'utilities');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('utilities', net, options.currency));
|
||||
totals.set('utilities', net);
|
||||
}
|
||||
}
|
||||
|
||||
// 10. rent
|
||||
{
|
||||
const gross = sumOf(ledger, 'rent');
|
||||
const held = pendingOf(ledger, 'rent');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('rent', net, options.currency));
|
||||
totals.set('rent', net);
|
||||
}
|
||||
}
|
||||
|
||||
// 11. insurance
|
||||
{
|
||||
const gross = sumOf(ledger, 'insurance');
|
||||
const held = pendingOf(ledger, 'insurance');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('insurance', net, options.currency));
|
||||
totals.set('insurance', net);
|
||||
}
|
||||
}
|
||||
|
||||
// 12. legal
|
||||
{
|
||||
const gross = sumOf(ledger, 'legal');
|
||||
const held = pendingOf(ledger, 'legal');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('legal', net, options.currency));
|
||||
totals.set('legal', net);
|
||||
}
|
||||
}
|
||||
|
||||
// 13. shipping
|
||||
{
|
||||
const gross = sumOf(ledger, 'shipping');
|
||||
const held = pendingOf(ledger, 'shipping');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('shipping', net, options.currency));
|
||||
totals.set('shipping', net);
|
||||
}
|
||||
}
|
||||
|
||||
// 14. hosting
|
||||
{
|
||||
const gross = sumOf(ledger, 'hosting');
|
||||
const held = pendingOf(ledger, 'hosting');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('hosting', net, options.currency));
|
||||
totals.set('hosting', net);
|
||||
}
|
||||
}
|
||||
|
||||
// 15. support
|
||||
{
|
||||
const gross = sumOf(ledger, 'support');
|
||||
const held = pendingOf(ledger, 'support');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('support', net, options.currency));
|
||||
totals.set('support', net);
|
||||
}
|
||||
}
|
||||
|
||||
// 16. recruiting
|
||||
{
|
||||
const gross = sumOf(ledger, 'recruiting');
|
||||
const held = pendingOf(ledger, 'recruiting');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('recruiting', net, options.currency));
|
||||
totals.set('recruiting', net);
|
||||
}
|
||||
}
|
||||
|
||||
// 17. licenses
|
||||
{
|
||||
const gross = sumOf(ledger, 'licenses');
|
||||
const held = pendingOf(ledger, 'licenses');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('licenses', net, options.currency));
|
||||
totals.set('licenses', net);
|
||||
}
|
||||
}
|
||||
|
||||
// 18. taxes
|
||||
{
|
||||
const gross = sumOf(ledger, 'taxes');
|
||||
const held = pendingOf(ledger, 'taxes');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('taxes', net, options.currency));
|
||||
totals.set('taxes', net);
|
||||
}
|
||||
}
|
||||
|
||||
// 19. refunds
|
||||
{
|
||||
const gross = sumOf(ledger, 'refunds');
|
||||
const held = pendingOf(ledger, 'refunds');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('refunds', net, options.currency));
|
||||
totals.set('refunds', net);
|
||||
}
|
||||
}
|
||||
|
||||
// 20. discounts
|
||||
{
|
||||
const gross = sumOf(ledger, 'discounts');
|
||||
const held = pendingOf(ledger, 'discounts');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('discounts', net, options.currency));
|
||||
totals.set('discounts', net);
|
||||
}
|
||||
}
|
||||
|
||||
// 21. interest
|
||||
{
|
||||
const gross = sumOf(ledger, 'interest');
|
||||
const held = pendingOf(ledger, 'interest');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('interest', net, options.currency));
|
||||
totals.set('interest', net);
|
||||
}
|
||||
}
|
||||
|
||||
// 22. depreciation
|
||||
{
|
||||
const gross = sumOf(ledger, 'depreciation');
|
||||
const held = pendingOf(ledger, 'depreciation');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('depreciation', net, options.currency));
|
||||
totals.set('depreciation', net);
|
||||
}
|
||||
}
|
||||
|
||||
// 23. maintenance
|
||||
{
|
||||
const gross = sumOf(ledger, 'maintenance');
|
||||
const held = pendingOf(ledger, 'maintenance');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('maintenance', net, options.currency));
|
||||
totals.set('maintenance', net);
|
||||
}
|
||||
}
|
||||
|
||||
// 24. subscriptions
|
||||
{
|
||||
const gross = sumOf(ledger, 'subscriptions');
|
||||
const held = pendingOf(ledger, 'subscriptions');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('subscriptions', net, options.currency));
|
||||
totals.set('subscriptions', net);
|
||||
}
|
||||
}
|
||||
|
||||
// 25. hardware
|
||||
{
|
||||
const gross = sumOf(ledger, 'hardware');
|
||||
const held = pendingOf(ledger, 'hardware');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('hardware', net, options.currency));
|
||||
totals.set('hardware', net);
|
||||
}
|
||||
}
|
||||
|
||||
// 26. catering
|
||||
{
|
||||
const gross = sumOf(ledger, 'catering');
|
||||
const held = pendingOf(ledger, 'catering');
|
||||
const net = options.includePending
|
||||
? gross
|
||||
: gross - held;
|
||||
if (net !== 0) {
|
||||
rows.push(formatReportRow('catering', net, options.currency));
|
||||
totals.set('catering', net);
|
||||
}
|
||||
}
|
||||
|
||||
const grandTotal = [...totals.values()]
|
||||
.reduce((sum, value) => sum + value, 0);
|
||||
rows.push(formatReportRow('total', grandTotal, options.currency));
|
||||
persistReport(ledger.periodId, rows);
|
||||
return rows;
|
||||
}
|
||||
|
||||
/** Header line for a rendered weekly report. */
|
||||
export function buildWeeklyReportHeader(ledger: Ledger, options: ReportOptions): string {
|
||||
return `weekly report ${ledger.periodId} (${options.currency})`;
|
||||
}
|
||||
@@ -0,0 +1,95 @@
|
||||
# payroll-go — the #1500 regression fixture
|
||||
|
||||
A synthetic Go service reproducing the repo shape from [issue #1500](https://github.com/colbymchenry/codegraph/issues/1500):
|
||||
**generated CRUD sitting beside the hand-written use-case that does the real work.**
|
||||
|
||||
This tree is a fixture, not a program. It never compiles or runs — it exists to be
|
||||
indexed. Keep it valid, idiomatic Go anyway: the extractor's output is the whole point.
|
||||
|
||||
## The shape
|
||||
|
||||
```
|
||||
cmd/payrolld/main.go wires the service
|
||||
internal/transport/httpapi/ HTTP entry point → use-case
|
||||
internal/usecase/payroll/ ← THE ANSWER. Hand-written workflow:
|
||||
cycle.go runPayrollCycleAll (227 lines)
|
||||
payslip_builder.go BuildPayslip — the actual pay calculation
|
||||
prorate.go
|
||||
internal/domain/payroll/payslip.go hand-written domain types
|
||||
internal/store/payslipstore/store.go the real Upsert
|
||||
internal/platform/clock/clock.go
|
||||
|
||||
internal/gen/fkit/payroll/ ← THE NOISE. Generated CRUD, ORDINARY names:
|
||||
payslip.go CreatePayslip, GetPayslip, UpdatePayslip, a second BuildPayslip
|
||||
payroll_cycle.go CreatePayrollCycle, PayrollCycleCreateRequest, …
|
||||
store.go a second Upsert
|
||||
calculate.go CalculatePayrollCycleTotals, CalculatePayslipNet, …
|
||||
dto.go
|
||||
internal/gen/fkit/employee/, timesheet/ more generated CRUD
|
||||
internal/gen/payrollpb/*.pb.go generated, detectable by PATH
|
||||
```
|
||||
|
||||
The chain the fixture is built around is `runPayrollCycleAll` → `BuildPayslip` → `Upsert`,
|
||||
entered from `POST /v1/payroll/cycles/{cycleID}/run`.
|
||||
|
||||
## The three properties that make it a regression fixture
|
||||
|
||||
1. **Generated files that only a CONTENT header betrays.** The `internal/gen/fkit/**`
|
||||
files have ordinary names (`payslip.go`, `store.go`) and carry
|
||||
`// Code generated by fkit v3.11.0. DO NOT EDIT.`. Path-only detection misses every
|
||||
one of them — that is the #1500 case, and why CG-5 added the content check. The
|
||||
`payrollpb/*.pb.go` files cover the path-detectable channel beside them.
|
||||
|
||||
2. **Deliberate name collisions.** `BuildPayslip`, `Upsert` and `Store` each exist twice,
|
||||
once generated and once hand-written. The generated layer also name-collides on every
|
||||
term of the question below — `CreatePayslip`, `PayrollCycleCreateRequest`,
|
||||
`CalculatePayrollCycleTotals` — so a scorer that rewards incidental name matches
|
||||
surfaces the CRUD path.
|
||||
|
||||
3. **A size split that drives the render mode.** `cycle.go` is deliberately over the
|
||||
whole-file window (227 lines) so it falls through to clipped clusters; the generated
|
||||
files are deliberately under it so they ship whole. Allocation follows file size, not
|
||||
relevance. `__tests__/explore-allocation-1500.test.ts` pins both sides of that split —
|
||||
if you edit these files, keep it.
|
||||
|
||||
## The assertion
|
||||
|
||||
Query: **"how does payroll cycle create and calculate payslips?"** — an architecture
|
||||
question that names none of the symbols that answer it. The budget should concentrate on
|
||||
the hand-written workflow. As of 2026-08-03 it does not:
|
||||
|
||||
| | allocated | delivered |
|
||||
|---|---|---|
|
||||
| hand-written | 48.4% | **25.6%** (all of it domain types) |
|
||||
| generated CRUD | 39.9% | **57.4%** |
|
||||
|
||||
`cycle.go` is allocated the single largest slice (7,052 chars, 30.6%) and delivers
|
||||
**zero** — the hard ceiling drops its whole section. `payslip_builder.go` (rank #8) never
|
||||
renders at all. `runPayrollCycleAll`, the hand-written `BuildPayslip` and the real
|
||||
`Upsert` never reach the agent.
|
||||
|
||||
## Running it
|
||||
|
||||
```bash
|
||||
npm run build
|
||||
node scripts/agent-eval/probe-allocation.mjs payroll-go # exits 1 today, by design
|
||||
npx vitest run __tests__/explore-allocation-1500.test.ts # green today, by design
|
||||
```
|
||||
|
||||
The probe reports per-file budget share against `scripts/agent-eval/allocation-fixtures.json`.
|
||||
The vitest suite pins the fixture's shape and holds the allocation assertion as `it.fails` —
|
||||
green while the bug is open, red the moment it is fixed. See
|
||||
`docs/design/explore-budget-allocation.md`.
|
||||
|
||||
## Known finding: the chain's `Upsert` edge resolves to the generated store
|
||||
|
||||
`runPayrollCycleAll` calls `s.store.Upsert(ctx, slip)`, where `s.store` is a
|
||||
`*payslipstore.Store`. The graph resolves that edge to `internal/gen/fkit/payroll/store.go`
|
||||
— the **generated** `Store.Upsert` — not the hand-written one. Same-name method resolution
|
||||
across two packages that both define `Store.Upsert` picks the wrong receiver.
|
||||
|
||||
This is a resolution defect, not a budget one, and it is left unfixed on purpose: it is
|
||||
upstream of the allocation bug (a wrong edge pulls the generated store into the subgraph
|
||||
and inflates its score), so it belongs with the scoring work in CG-10 rather than here.
|
||||
The test asserts only that the workflow reaches *an* `Upsert`, so tightening the resolver
|
||||
later will not break the fixture.
|
||||
@@ -0,0 +1,35 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"log"
|
||||
"net/http"
|
||||
"os"
|
||||
"time"
|
||||
|
||||
"github.com/example/payroll-svc/internal/platform/clock"
|
||||
"github.com/example/payroll-svc/internal/store/payslipstore"
|
||||
"github.com/example/payroll-svc/internal/transport/httpapi"
|
||||
"github.com/example/payroll-svc/internal/usecase/payroll"
|
||||
)
|
||||
|
||||
func main() {
|
||||
addr := os.Getenv("LISTEN_ADDR")
|
||||
if addr == "" {
|
||||
addr = ":8080"
|
||||
}
|
||||
|
||||
store := payslipstore.New()
|
||||
svc := payroll.NewService(store, clock.System{})
|
||||
router := httpapi.NewRouter(httpapi.NewPayrollHandler(svc))
|
||||
|
||||
srv := &http.Server{
|
||||
Addr: addr,
|
||||
Handler: router,
|
||||
ReadHeaderTimeout: 5 * time.Second,
|
||||
}
|
||||
|
||||
log.Printf("payrolld listening on %s", addr)
|
||||
if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
|
||||
log.Fatalf("payrolld: %v", err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
module github.com/example/payroll-svc
|
||||
|
||||
go 1.22
|
||||
@@ -0,0 +1,159 @@
|
||||
package payroll
|
||||
|
||||
import "time"
|
||||
|
||||
// CycleStatus is the lifecycle state of a payroll cycle.
|
||||
type CycleStatus string
|
||||
|
||||
const (
|
||||
CycleOpen CycleStatus = "open"
|
||||
CycleClosed CycleStatus = "closed"
|
||||
)
|
||||
|
||||
// ContractKind distinguishes the two pay models this service supports.
|
||||
type ContractKind string
|
||||
|
||||
const (
|
||||
ContractSalaried ContractKind = "salaried"
|
||||
ContractHourly ContractKind = "hourly"
|
||||
)
|
||||
|
||||
// LineKind separates the two halves of a payslip.
|
||||
type LineKind string
|
||||
|
||||
const (
|
||||
LineEarning LineKind = "earning"
|
||||
LineDeduction LineKind = "deduction"
|
||||
)
|
||||
|
||||
// Cycle is one payroll period.
|
||||
type Cycle struct {
|
||||
ID string
|
||||
Start time.Time
|
||||
End time.Time
|
||||
Status CycleStatus
|
||||
ClosedAt time.Time
|
||||
ReopenReason string
|
||||
}
|
||||
|
||||
// Line is a single earning or deduction on a payslip.
|
||||
type Line struct {
|
||||
Code string
|
||||
Kind LineKind
|
||||
AmountCents int64
|
||||
}
|
||||
|
||||
// Payslip is what a cycle produces for one employee.
|
||||
type Payslip struct {
|
||||
CycleID string
|
||||
EmployeeID string
|
||||
Currency string
|
||||
PeriodFrom time.Time
|
||||
PeriodTo time.Time
|
||||
Lines []Line
|
||||
GrossCents int64
|
||||
DeductionCents int64
|
||||
NetCents int64
|
||||
Underwater bool
|
||||
RunAt time.Time
|
||||
RunReason string
|
||||
}
|
||||
|
||||
// Timesheet is the approved unit count backing an hourly payslip.
|
||||
type Timesheet struct {
|
||||
CycleID string
|
||||
EmployeeID string
|
||||
Approved bool
|
||||
Units int
|
||||
}
|
||||
|
||||
// Allowance is a recurring earning attached to a contract.
|
||||
type Allowance struct {
|
||||
Code string
|
||||
AmountCents int64
|
||||
Prorated bool
|
||||
}
|
||||
|
||||
// Contract holds the pay terms for one employee.
|
||||
type Contract struct {
|
||||
Kind ContractKind
|
||||
Currency string
|
||||
RateCents int64
|
||||
PeriodRateCents int64
|
||||
OvertimeThresholdUnits int
|
||||
OvertimeMultiplier float64
|
||||
Allowances []Allowance
|
||||
StartsOn time.Time
|
||||
EndsOn time.Time
|
||||
}
|
||||
|
||||
// OverlapsWindow reports whether the contract is live at any point in the window.
|
||||
func (c Contract) OverlapsWindow(from, to time.Time) bool {
|
||||
if !c.StartsOn.IsZero() && c.StartsOn.After(to) {
|
||||
return false
|
||||
}
|
||||
if !c.EndsOn.IsZero() && c.EndsOn.Before(from) {
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// PeriodUnits is the contractual unit count for a window, used when a salaried
|
||||
// employee has no approved timesheet.
|
||||
func (c Contract) PeriodUnits(from, to time.Time) int {
|
||||
if to.Before(from) {
|
||||
return 0
|
||||
}
|
||||
days := int(to.Sub(from).Hours()/24) + 1
|
||||
return days * 8
|
||||
}
|
||||
|
||||
// Deduction is a fixed or proportional subtraction from gross.
|
||||
type Deduction struct {
|
||||
Code string
|
||||
FixedCents int64
|
||||
RateBasisPoints int
|
||||
}
|
||||
|
||||
// AmountFor resolves a deduction against a gross amount.
|
||||
func (d Deduction) AmountFor(grossCents int64) int64 {
|
||||
if d.FixedCents > 0 {
|
||||
return d.FixedCents
|
||||
}
|
||||
return grossCents * int64(d.RateBasisPoints) / 10000
|
||||
}
|
||||
|
||||
// TaxBand is one slice of a progressive tax schedule.
|
||||
type TaxBand struct {
|
||||
UpToCents int64
|
||||
RateBasisPoints int
|
||||
}
|
||||
|
||||
// Leave is an absence window.
|
||||
type Leave struct {
|
||||
From time.Time
|
||||
To time.Time
|
||||
Unpaid bool
|
||||
}
|
||||
|
||||
// Employee is the payroll view of a person.
|
||||
type Employee struct {
|
||||
ID string
|
||||
Contract Contract
|
||||
Deductions []Deduction
|
||||
TaxBands []TaxBand
|
||||
Leave []Leave
|
||||
}
|
||||
|
||||
// UnpaidLeaveCoversWindow reports whether unpaid leave swallows the whole window.
|
||||
func (e Employee) UnpaidLeaveCoversWindow(from, to time.Time) bool {
|
||||
for _, l := range e.Leave {
|
||||
if !l.Unpaid {
|
||||
continue
|
||||
}
|
||||
if !l.From.After(from) && !l.To.Before(to) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
// Code generated by fkit v3.11.0. DO NOT EDIT.
|
||||
//
|
||||
// Source: schema/employee/contract.fkit
|
||||
// Regenerate with: go run ./tools/fkitgen ./schema/employee
|
||||
|
||||
package employee
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"time"
|
||||
)
|
||||
|
||||
// ContractRow is the generated row type for table contract.
|
||||
type ContractRow struct {
|
||||
ID string
|
||||
EmployeeID string
|
||||
Kind string
|
||||
Currency string
|
||||
RateCents int64
|
||||
PeriodRateCents int64
|
||||
StartsOn time.Time
|
||||
EndsOn time.Time
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
}
|
||||
|
||||
// ContractCreateRequest is the generated create payload for table contract.
|
||||
type ContractCreateRequest struct {
|
||||
EmployeeID string `json:"employeeId"`
|
||||
Kind string `json:"kind"`
|
||||
Currency string `json:"currency"`
|
||||
RateCents int64 `json:"rateCents"`
|
||||
PeriodRateCents int64 `json:"periodRateCents"`
|
||||
StartsOn time.Time `json:"startsOn"`
|
||||
}
|
||||
|
||||
// CreateContract inserts one contract row.
|
||||
func CreateContract(ctx context.Context, db *sql.DB, req ContractCreateRequest) (ContractRow, error) {
|
||||
const q = `INSERT INTO contract (employee_id, kind, currency, rate_cents, period_rate_cents, starts_on)
|
||||
VALUES ($1, $2, $3, $4, $5, $6) RETURNING *`
|
||||
return scanContract(db.QueryRowContext(ctx, q, req.EmployeeID, req.Kind, req.Currency,
|
||||
req.RateCents, req.PeriodRateCents, req.StartsOn))
|
||||
}
|
||||
|
||||
// GetContract selects one contract row by primary key.
|
||||
func GetContract(ctx context.Context, db *sql.DB, id string) (ContractRow, error) {
|
||||
const q = `SELECT * FROM contract WHERE id = $1`
|
||||
return scanContract(db.QueryRowContext(ctx, q, id))
|
||||
}
|
||||
|
||||
// DeleteContract removes one contract row.
|
||||
func DeleteContract(ctx context.Context, db *sql.DB, id string) error {
|
||||
const q = `DELETE FROM contract WHERE id = $1`
|
||||
_, err := db.ExecContext(ctx, q, id)
|
||||
return err
|
||||
}
|
||||
|
||||
// ListContractsForEmployee selects contract rows for one employee.
|
||||
func ListContractsForEmployee(ctx context.Context, db *sql.DB, employeeID string) ([]ContractRow, error) {
|
||||
const q = `SELECT * FROM contract WHERE employee_id = $1 ORDER BY starts_on DESC`
|
||||
rows, err := db.QueryContext(ctx, q, employeeID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var out []ContractRow
|
||||
for rows.Next() {
|
||||
var r ContractRow
|
||||
if err := rows.Scan(&r.ID, &r.EmployeeID, &r.Kind, &r.Currency, &r.RateCents,
|
||||
&r.PeriodRateCents, &r.StartsOn, &r.EndsOn, &r.CreatedAt, &r.UpdatedAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, r)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
func scanContract(row *sql.Row) (ContractRow, error) {
|
||||
var r ContractRow
|
||||
err := row.Scan(&r.ID, &r.EmployeeID, &r.Kind, &r.Currency, &r.RateCents,
|
||||
&r.PeriodRateCents, &r.StartsOn, &r.EndsOn, &r.CreatedAt, &r.UpdatedAt)
|
||||
return r, err
|
||||
}
|
||||
@@ -0,0 +1,88 @@
|
||||
// Code generated by fkit v3.11.0. DO NOT EDIT.
|
||||
//
|
||||
// Source: schema/employee/employee.fkit
|
||||
// Regenerate with: go run ./tools/fkitgen ./schema/employee
|
||||
|
||||
package employee
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"time"
|
||||
)
|
||||
|
||||
// EmployeeRow is the generated row type for table employee.
|
||||
type EmployeeRow struct {
|
||||
ID string
|
||||
Email string
|
||||
FullName string
|
||||
Status string
|
||||
HiredOn time.Time
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
}
|
||||
|
||||
// EmployeeCreateRequest is the generated create payload for table employee.
|
||||
type EmployeeCreateRequest struct {
|
||||
Email string `json:"email"`
|
||||
FullName string `json:"fullName"`
|
||||
Status string `json:"status"`
|
||||
}
|
||||
|
||||
// EmployeeUpdateRequest is the generated update payload for table employee.
|
||||
type EmployeeUpdateRequest struct {
|
||||
FullName *string `json:"fullName,omitempty"`
|
||||
Status *string `json:"status,omitempty"`
|
||||
}
|
||||
|
||||
// CreateEmployee inserts one employee row.
|
||||
func CreateEmployee(ctx context.Context, db *sql.DB, req EmployeeCreateRequest) (EmployeeRow, error) {
|
||||
const q = `INSERT INTO employee (email, full_name, status) VALUES ($1, $2, $3) RETURNING *`
|
||||
return scanEmployee(db.QueryRowContext(ctx, q, req.Email, req.FullName, req.Status))
|
||||
}
|
||||
|
||||
// GetEmployee selects one employee row by primary key.
|
||||
func GetEmployee(ctx context.Context, db *sql.DB, id string) (EmployeeRow, error) {
|
||||
const q = `SELECT * FROM employee WHERE id = $1`
|
||||
return scanEmployee(db.QueryRowContext(ctx, q, id))
|
||||
}
|
||||
|
||||
// UpdateEmployee patches one employee row.
|
||||
func UpdateEmployee(ctx context.Context, db *sql.DB, id string, req EmployeeUpdateRequest) (EmployeeRow, error) {
|
||||
const q = `UPDATE employee SET full_name = COALESCE($2, full_name), status = COALESCE($3, status),
|
||||
updated_at = now() WHERE id = $1 RETURNING *`
|
||||
return scanEmployee(db.QueryRowContext(ctx, q, id, req.FullName, req.Status))
|
||||
}
|
||||
|
||||
// DeleteEmployee removes one employee row.
|
||||
func DeleteEmployee(ctx context.Context, db *sql.DB, id string) error {
|
||||
const q = `DELETE FROM employee WHERE id = $1`
|
||||
_, err := db.ExecContext(ctx, q, id)
|
||||
return err
|
||||
}
|
||||
|
||||
// ListEmployeesByStatus selects employee rows in one status.
|
||||
func ListEmployeesByStatus(ctx context.Context, db *sql.DB, status string) ([]EmployeeRow, error) {
|
||||
const q = `SELECT * FROM employee WHERE status = $1 ORDER BY full_name`
|
||||
rows, err := db.QueryContext(ctx, q, status)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var out []EmployeeRow
|
||||
for rows.Next() {
|
||||
var r EmployeeRow
|
||||
if err := rows.Scan(&r.ID, &r.Email, &r.FullName, &r.Status, &r.HiredOn,
|
||||
&r.CreatedAt, &r.UpdatedAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, r)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
func scanEmployee(row *sql.Row) (EmployeeRow, error) {
|
||||
var r EmployeeRow
|
||||
err := row.Scan(&r.ID, &r.Email, &r.FullName, &r.Status, &r.HiredOn, &r.CreatedAt, &r.UpdatedAt)
|
||||
return r, err
|
||||
}
|
||||
@@ -0,0 +1,60 @@
|
||||
// Code generated by fkit v3.11.0. DO NOT EDIT.
|
||||
//
|
||||
// Source: schema/payroll/aggregates.fkit
|
||||
// Regenerate with: go run ./tools/fkitgen ./schema/payroll
|
||||
|
||||
package payroll
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
)
|
||||
|
||||
// PayrollCycleTotals is the generated aggregate row for a payroll cycle.
|
||||
type PayrollCycleTotals struct {
|
||||
CycleID string
|
||||
Payslips int64
|
||||
GrossCents int64
|
||||
DeductionCents int64
|
||||
NetCents int64
|
||||
}
|
||||
|
||||
// CalculatePayrollCycleTotals runs the generated SUM aggregate over the
|
||||
// payslip rows of one cycle. It totals what is already stored; it does not
|
||||
// calculate any payslip.
|
||||
func CalculatePayrollCycleTotals(ctx context.Context, db *sql.DB, cycleID string) (PayrollCycleTotals, error) {
|
||||
const q = `SELECT count(*), COALESCE(sum(gross_cents), 0), COALESCE(sum(deduction_cents), 0),
|
||||
COALESCE(sum(net_cents), 0)
|
||||
FROM payslip WHERE cycle_id = $1`
|
||||
var t PayrollCycleTotals
|
||||
t.CycleID = cycleID
|
||||
err := db.QueryRowContext(ctx, q, cycleID).Scan(&t.Payslips, &t.GrossCents, &t.DeductionCents, &t.NetCents)
|
||||
return t, err
|
||||
}
|
||||
|
||||
// CalculatePayslipNet recomputes net from the stored gross and deduction
|
||||
// columns of one row. Pure column arithmetic — no pay rules.
|
||||
func CalculatePayslipNet(row PayslipRow) int64 {
|
||||
return row.GrossCents - row.DeductionCents
|
||||
}
|
||||
|
||||
// CalculatePayrollCycleAverage averages the stored net over a cycle.
|
||||
func CalculatePayrollCycleAverage(ctx context.Context, db *sql.DB, cycleID string) (int64, error) {
|
||||
totals, err := CalculatePayrollCycleTotals(ctx, db, cycleID)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if totals.Payslips == 0 {
|
||||
return 0, nil
|
||||
}
|
||||
return totals.NetCents / totals.Payslips, nil
|
||||
}
|
||||
|
||||
// CalculateEmployeeYearToDate sums an employee's stored payslips for a year.
|
||||
func CalculateEmployeeYearToDate(ctx context.Context, db *sql.DB, employeeID string, year int) (int64, error) {
|
||||
const q = `SELECT COALESCE(sum(net_cents), 0) FROM payslip
|
||||
WHERE employee_id = $1 AND extract(year from period_from) = $2`
|
||||
var n int64
|
||||
err := db.QueryRowContext(ctx, q, employeeID, year).Scan(&n)
|
||||
return n, err
|
||||
}
|
||||
@@ -0,0 +1,80 @@
|
||||
// Code generated by fkit v3.11.0. DO NOT EDIT.
|
||||
//
|
||||
// Source: schema/payroll
|
||||
// Regenerate with: go run ./tools/fkitgen ./schema/payroll
|
||||
|
||||
package payroll
|
||||
|
||||
import "time"
|
||||
|
||||
// PayslipDTO is the generated wire representation of a payslip row.
|
||||
type PayslipDTO struct {
|
||||
ID string `json:"id"`
|
||||
CycleID string `json:"cycleId"`
|
||||
EmployeeID string `json:"employeeId"`
|
||||
Currency string `json:"currency"`
|
||||
PeriodFrom time.Time `json:"periodFrom"`
|
||||
PeriodTo time.Time `json:"periodTo"`
|
||||
GrossCents int64 `json:"grossCents"`
|
||||
DeductionCents int64 `json:"deductionCents"`
|
||||
NetCents int64 `json:"netCents"`
|
||||
}
|
||||
|
||||
// PayrollCycleDTO is the generated wire representation of a payroll_cycle row.
|
||||
type PayrollCycleDTO struct {
|
||||
ID string `json:"id"`
|
||||
Start time.Time `json:"start"`
|
||||
End time.Time `json:"end"`
|
||||
Status string `json:"status"`
|
||||
}
|
||||
|
||||
// PayslipListDTO is the generated list envelope for payslip rows.
|
||||
type PayslipListDTO struct {
|
||||
Items []PayslipDTO `json:"items"`
|
||||
NextCursor string `json:"nextCursor,omitempty"`
|
||||
Total int64 `json:"total"`
|
||||
}
|
||||
|
||||
// PayslipToDTO converts a payslip row to its wire form.
|
||||
func PayslipToDTO(r PayslipRow) PayslipDTO {
|
||||
return PayslipDTO{
|
||||
ID: r.ID,
|
||||
CycleID: r.CycleID,
|
||||
EmployeeID: r.EmployeeID,
|
||||
Currency: r.Currency,
|
||||
PeriodFrom: r.PeriodFrom,
|
||||
PeriodTo: r.PeriodTo,
|
||||
GrossCents: r.GrossCents,
|
||||
DeductionCents: r.DeductionCents,
|
||||
NetCents: r.NetCents,
|
||||
}
|
||||
}
|
||||
|
||||
// PayslipFromDTO converts a wire payslip back to a row.
|
||||
func PayslipFromDTO(d PayslipDTO) PayslipRow {
|
||||
return PayslipRow{
|
||||
ID: d.ID,
|
||||
CycleID: d.CycleID,
|
||||
EmployeeID: d.EmployeeID,
|
||||
Currency: d.Currency,
|
||||
PeriodFrom: d.PeriodFrom,
|
||||
PeriodTo: d.PeriodTo,
|
||||
GrossCents: d.GrossCents,
|
||||
DeductionCents: d.DeductionCents,
|
||||
NetCents: d.NetCents,
|
||||
}
|
||||
}
|
||||
|
||||
// PayrollCycleToDTO converts a payroll_cycle row to its wire form.
|
||||
func PayrollCycleToDTO(r PayrollCycleRow) PayrollCycleDTO {
|
||||
return PayrollCycleDTO{ID: r.ID, Start: r.Start, End: r.End, Status: r.Status}
|
||||
}
|
||||
|
||||
// PayslipsToListDTO wraps payslip rows in the generated list envelope.
|
||||
func PayslipsToListDTO(rows []PayslipRow, total int64) PayslipListDTO {
|
||||
items := make([]PayslipDTO, 0, len(rows))
|
||||
for _, r := range rows {
|
||||
items = append(items, PayslipToDTO(r))
|
||||
}
|
||||
return PayslipListDTO{Items: items, Total: total}
|
||||
}
|
||||
@@ -0,0 +1,116 @@
|
||||
// Code generated by fkit v3.11.0. DO NOT EDIT.
|
||||
//
|
||||
// Source: schema/payroll/payroll_cycle.fkit
|
||||
// Regenerate with: go run ./tools/fkitgen ./schema/payroll
|
||||
|
||||
package payroll
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"time"
|
||||
)
|
||||
|
||||
// PayrollCycleRow is the generated row type for table payroll_cycle.
|
||||
type PayrollCycleRow struct {
|
||||
ID string
|
||||
Start time.Time
|
||||
End time.Time
|
||||
Status string
|
||||
ClosedAt time.Time
|
||||
ReopenReason string
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
}
|
||||
|
||||
// PayrollCycleCreateRequest is the generated create payload for payroll_cycle.
|
||||
type PayrollCycleCreateRequest struct {
|
||||
Start time.Time `json:"start"`
|
||||
End time.Time `json:"end"`
|
||||
Status string `json:"status"`
|
||||
}
|
||||
|
||||
// PayrollCycleUpdateRequest is the generated update payload for payroll_cycle.
|
||||
type PayrollCycleUpdateRequest struct {
|
||||
Status *string `json:"status,omitempty"`
|
||||
ReopenReason *string `json:"reopenReason,omitempty"`
|
||||
}
|
||||
|
||||
// CreatePayrollCycle inserts one payroll_cycle row.
|
||||
func CreatePayrollCycle(ctx context.Context, db *sql.DB, req PayrollCycleCreateRequest) (PayrollCycleRow, error) {
|
||||
const q = `INSERT INTO payroll_cycle (start_on, end_on, status) VALUES ($1, $2, $3) RETURNING *`
|
||||
return scanPayrollCycle(db.QueryRowContext(ctx, q, req.Start, req.End, req.Status))
|
||||
}
|
||||
|
||||
// GetPayrollCycle selects one payroll_cycle row by primary key.
|
||||
func GetPayrollCycle(ctx context.Context, db *sql.DB, id string) (PayrollCycleRow, error) {
|
||||
const q = `SELECT * FROM payroll_cycle WHERE id = $1`
|
||||
return scanPayrollCycle(db.QueryRowContext(ctx, q, id))
|
||||
}
|
||||
|
||||
// UpdatePayrollCycle patches one payroll_cycle row.
|
||||
func UpdatePayrollCycle(ctx context.Context, db *sql.DB, id string, req PayrollCycleUpdateRequest) (PayrollCycleRow, error) {
|
||||
const q = `UPDATE payroll_cycle SET status = COALESCE($2, status),
|
||||
reopen_reason = COALESCE($3, reopen_reason), updated_at = now()
|
||||
WHERE id = $1 RETURNING *`
|
||||
return scanPayrollCycle(db.QueryRowContext(ctx, q, id, req.Status, req.ReopenReason))
|
||||
}
|
||||
|
||||
// DeletePayrollCycle removes one payroll_cycle row.
|
||||
func DeletePayrollCycle(ctx context.Context, db *sql.DB, id string) error {
|
||||
const q = `DELETE FROM payroll_cycle WHERE id = $1`
|
||||
_, err := db.ExecContext(ctx, q, id)
|
||||
return err
|
||||
}
|
||||
|
||||
// ListPayrollCycles selects every payroll_cycle row.
|
||||
func ListPayrollCycles(ctx context.Context, db *sql.DB) ([]PayrollCycleRow, error) {
|
||||
const q = `SELECT * FROM payroll_cycle ORDER BY start_on DESC`
|
||||
rows, err := db.QueryContext(ctx, q)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var out []PayrollCycleRow
|
||||
for rows.Next() {
|
||||
var r PayrollCycleRow
|
||||
if err := rows.Scan(&r.ID, &r.Start, &r.End, &r.Status, &r.ClosedAt, &r.ReopenReason,
|
||||
&r.CreatedAt, &r.UpdatedAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, r)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
// ListPayrollCyclesByStatus selects payroll_cycle rows in one status.
|
||||
func ListPayrollCyclesByStatus(ctx context.Context, db *sql.DB, status string) ([]PayrollCycleRow, error) {
|
||||
const q = `SELECT * FROM payroll_cycle WHERE status = $1 ORDER BY start_on DESC`
|
||||
rows, err := db.QueryContext(ctx, q, status)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var out []PayrollCycleRow
|
||||
for rows.Next() {
|
||||
var r PayrollCycleRow
|
||||
if err := rows.Scan(&r.ID, &r.Start, &r.End, &r.Status, &r.ClosedAt, &r.ReopenReason,
|
||||
&r.CreatedAt, &r.UpdatedAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, r)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
// BuildPayrollCycle maps a create request onto a row.
|
||||
func BuildPayrollCycle(req PayrollCycleCreateRequest) PayrollCycleRow {
|
||||
return PayrollCycleRow{Start: req.Start, End: req.End, Status: req.Status}
|
||||
}
|
||||
|
||||
func scanPayrollCycle(row *sql.Row) (PayrollCycleRow, error) {
|
||||
var r PayrollCycleRow
|
||||
err := row.Scan(&r.ID, &r.Start, &r.End, &r.Status, &r.ClosedAt, &r.ReopenReason,
|
||||
&r.CreatedAt, &r.UpdatedAt)
|
||||
return r, err
|
||||
}
|
||||
@@ -0,0 +1,129 @@
|
||||
// Code generated by fkit v3.11.0. DO NOT EDIT.
|
||||
//
|
||||
// Source: schema/payroll/payslip.fkit
|
||||
// Regenerate with: go run ./tools/fkitgen ./schema/payroll
|
||||
|
||||
package payroll
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"time"
|
||||
)
|
||||
|
||||
// PayslipRow is the generated row type for table payslip.
|
||||
type PayslipRow struct {
|
||||
ID string
|
||||
CycleID string
|
||||
EmployeeID string
|
||||
Currency string
|
||||
PeriodFrom time.Time
|
||||
PeriodTo time.Time
|
||||
GrossCents int64
|
||||
DeductionCents int64
|
||||
NetCents int64
|
||||
Underwater bool
|
||||
RunAt time.Time
|
||||
RunReason string
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
}
|
||||
|
||||
// PayslipCreateRequest is the generated create payload for table payslip.
|
||||
type PayslipCreateRequest struct {
|
||||
CycleID string `json:"cycleId"`
|
||||
EmployeeID string `json:"employeeId"`
|
||||
Currency string `json:"currency"`
|
||||
GrossCents int64 `json:"grossCents"`
|
||||
DeductionCents int64 `json:"deductionCents"`
|
||||
NetCents int64 `json:"netCents"`
|
||||
}
|
||||
|
||||
// PayslipUpdateRequest is the generated update payload for table payslip.
|
||||
type PayslipUpdateRequest struct {
|
||||
GrossCents *int64 `json:"grossCents,omitempty"`
|
||||
DeductionCents *int64 `json:"deductionCents,omitempty"`
|
||||
NetCents *int64 `json:"netCents,omitempty"`
|
||||
RunReason *string `json:"runReason,omitempty"`
|
||||
}
|
||||
|
||||
// CreatePayslip inserts one payslip row.
|
||||
func CreatePayslip(ctx context.Context, db *sql.DB, req PayslipCreateRequest) (PayslipRow, error) {
|
||||
const q = `INSERT INTO payslip (cycle_id, employee_id, currency, gross_cents, deduction_cents, net_cents)
|
||||
VALUES ($1, $2, $3, $4, $5, $6) RETURNING *`
|
||||
row := db.QueryRowContext(ctx, q, req.CycleID, req.EmployeeID, req.Currency, req.GrossCents, req.DeductionCents, req.NetCents)
|
||||
return scanPayslip(row)
|
||||
}
|
||||
|
||||
// GetPayslip selects one payslip row by primary key.
|
||||
func GetPayslip(ctx context.Context, db *sql.DB, id string) (PayslipRow, error) {
|
||||
const q = `SELECT * FROM payslip WHERE id = $1`
|
||||
return scanPayslip(db.QueryRowContext(ctx, q, id))
|
||||
}
|
||||
|
||||
// UpdatePayslip patches one payslip row.
|
||||
func UpdatePayslip(ctx context.Context, db *sql.DB, id string, req PayslipUpdateRequest) (PayslipRow, error) {
|
||||
const q = `UPDATE payslip SET gross_cents = COALESCE($2, gross_cents),
|
||||
deduction_cents = COALESCE($3, deduction_cents),
|
||||
net_cents = COALESCE($4, net_cents),
|
||||
run_reason = COALESCE($5, run_reason),
|
||||
updated_at = now() WHERE id = $1 RETURNING *`
|
||||
return scanPayslip(db.QueryRowContext(ctx, q, id, req.GrossCents, req.DeductionCents, req.NetCents, req.RunReason))
|
||||
}
|
||||
|
||||
// DeletePayslip removes one payslip row.
|
||||
func DeletePayslip(ctx context.Context, db *sql.DB, id string) error {
|
||||
const q = `DELETE FROM payslip WHERE id = $1`
|
||||
_, err := db.ExecContext(ctx, q, id)
|
||||
return err
|
||||
}
|
||||
|
||||
// ListPayslipsByCycle selects every payslip row for a cycle.
|
||||
func ListPayslipsByCycle(ctx context.Context, db *sql.DB, cycleID string) ([]PayslipRow, error) {
|
||||
const q = `SELECT * FROM payslip WHERE cycle_id = $1 ORDER BY employee_id`
|
||||
rows, err := db.QueryContext(ctx, q, cycleID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var out []PayslipRow
|
||||
for rows.Next() {
|
||||
var r PayslipRow
|
||||
if err := rows.Scan(&r.ID, &r.CycleID, &r.EmployeeID, &r.Currency, &r.PeriodFrom, &r.PeriodTo,
|
||||
&r.GrossCents, &r.DeductionCents, &r.NetCents, &r.Underwater, &r.RunAt, &r.RunReason,
|
||||
&r.CreatedAt, &r.UpdatedAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, r)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
// CountPayslipsByCycle counts payslip rows for a cycle.
|
||||
func CountPayslipsByCycle(ctx context.Context, db *sql.DB, cycleID string) (int64, error) {
|
||||
const q = `SELECT count(*) FROM payslip WHERE cycle_id = $1`
|
||||
var n int64
|
||||
err := db.QueryRowContext(ctx, q, cycleID).Scan(&n)
|
||||
return n, err
|
||||
}
|
||||
|
||||
// BuildPayslip maps a create request onto a row. Field copy only — the
|
||||
// generator has no knowledge of pay rules.
|
||||
func BuildPayslip(req PayslipCreateRequest) PayslipRow {
|
||||
return PayslipRow{
|
||||
CycleID: req.CycleID,
|
||||
EmployeeID: req.EmployeeID,
|
||||
Currency: req.Currency,
|
||||
GrossCents: req.GrossCents,
|
||||
DeductionCents: req.DeductionCents,
|
||||
NetCents: req.NetCents,
|
||||
}
|
||||
}
|
||||
|
||||
func scanPayslip(row *sql.Row) (PayslipRow, error) {
|
||||
var r PayslipRow
|
||||
err := row.Scan(&r.ID, &r.CycleID, &r.EmployeeID, &r.Currency, &r.PeriodFrom, &r.PeriodTo,
|
||||
&r.GrossCents, &r.DeductionCents, &r.NetCents, &r.Underwater, &r.RunAt, &r.RunReason,
|
||||
&r.CreatedAt, &r.UpdatedAt)
|
||||
return r, err
|
||||
}
|
||||
@@ -0,0 +1,95 @@
|
||||
// Code generated by fkit v3.11.0. DO NOT EDIT.
|
||||
//
|
||||
// Source: schema/payroll
|
||||
// Regenerate with: go run ./tools/fkitgen ./schema/payroll
|
||||
|
||||
package payroll
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
)
|
||||
|
||||
// Store is the generated repository over every payroll table.
|
||||
type Store struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
// NewStore returns a generated store bound to db.
|
||||
func NewStore(db *sql.DB) *Store { return &Store{db: db} }
|
||||
|
||||
// Upsert writes one payslip row, keyed by (cycle_id, employee_id).
|
||||
func (s *Store) Upsert(ctx context.Context, row PayslipRow) (PayslipRow, error) {
|
||||
const q = `INSERT INTO payslip (cycle_id, employee_id, currency, gross_cents, deduction_cents, net_cents)
|
||||
VALUES ($1, $2, $3, $4, $5, $6)
|
||||
ON CONFLICT (cycle_id, employee_id) DO UPDATE SET
|
||||
gross_cents = EXCLUDED.gross_cents,
|
||||
deduction_cents = EXCLUDED.deduction_cents,
|
||||
net_cents = EXCLUDED.net_cents,
|
||||
updated_at = now()
|
||||
RETURNING *`
|
||||
return scanPayslip(s.db.QueryRowContext(ctx, q, row.CycleID, row.EmployeeID, row.Currency,
|
||||
row.GrossCents, row.DeductionCents, row.NetCents))
|
||||
}
|
||||
|
||||
// UpsertPayrollCycle writes one payroll_cycle row, keyed by id.
|
||||
func (s *Store) UpsertPayrollCycle(ctx context.Context, row PayrollCycleRow) (PayrollCycleRow, error) {
|
||||
const q = `INSERT INTO payroll_cycle (id, start_on, end_on, status)
|
||||
VALUES ($1, $2, $3, $4)
|
||||
ON CONFLICT (id) DO UPDATE SET status = EXCLUDED.status, updated_at = now()
|
||||
RETURNING *`
|
||||
return scanPayrollCycle(s.db.QueryRowContext(ctx, q, row.ID, row.Start, row.End, row.Status))
|
||||
}
|
||||
|
||||
// CreatePayslip inserts one payslip row through the store.
|
||||
func (s *Store) CreatePayslip(ctx context.Context, req PayslipCreateRequest) (PayslipRow, error) {
|
||||
return CreatePayslip(ctx, s.db, req)
|
||||
}
|
||||
|
||||
// GetPayslip reads one payslip row through the store.
|
||||
func (s *Store) GetPayslip(ctx context.Context, id string) (PayslipRow, error) {
|
||||
return GetPayslip(ctx, s.db, id)
|
||||
}
|
||||
|
||||
// UpdatePayslip patches one payslip row through the store.
|
||||
func (s *Store) UpdatePayslip(ctx context.Context, id string, req PayslipUpdateRequest) (PayslipRow, error) {
|
||||
return UpdatePayslip(ctx, s.db, id, req)
|
||||
}
|
||||
|
||||
// DeletePayslip removes one payslip row through the store.
|
||||
func (s *Store) DeletePayslip(ctx context.Context, id string) error {
|
||||
return DeletePayslip(ctx, s.db, id)
|
||||
}
|
||||
|
||||
// ListPayslipsByCycle lists payslip rows for a cycle through the store.
|
||||
func (s *Store) ListPayslipsByCycle(ctx context.Context, cycleID string) ([]PayslipRow, error) {
|
||||
return ListPayslipsByCycle(ctx, s.db, cycleID)
|
||||
}
|
||||
|
||||
// CreatePayrollCycle inserts one payroll_cycle row through the store.
|
||||
func (s *Store) CreatePayrollCycle(ctx context.Context, req PayrollCycleCreateRequest) (PayrollCycleRow, error) {
|
||||
return CreatePayrollCycle(ctx, s.db, req)
|
||||
}
|
||||
|
||||
// GetPayrollCycle reads one payroll_cycle row through the store.
|
||||
func (s *Store) GetPayrollCycle(ctx context.Context, id string) (PayrollCycleRow, error) {
|
||||
return GetPayrollCycle(ctx, s.db, id)
|
||||
}
|
||||
|
||||
// ListPayrollCycles lists payroll_cycle rows through the store.
|
||||
func (s *Store) ListPayrollCycles(ctx context.Context) ([]PayrollCycleRow, error) {
|
||||
return ListPayrollCycles(ctx, s.db)
|
||||
}
|
||||
|
||||
// Tx runs fn inside a transaction.
|
||||
func (s *Store) Tx(ctx context.Context, fn func(*Store) error) error {
|
||||
tx, err := s.db.BeginTx(ctx, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := fn(s); err != nil {
|
||||
_ = tx.Rollback()
|
||||
return err
|
||||
}
|
||||
return tx.Commit()
|
||||
}
|
||||
@@ -0,0 +1,76 @@
|
||||
// Code generated by fkit v3.11.0. DO NOT EDIT.
|
||||
//
|
||||
// Source: schema/timesheet/timesheet.fkit
|
||||
// Regenerate with: go run ./tools/fkitgen ./schema/timesheet
|
||||
|
||||
package timesheet
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"time"
|
||||
)
|
||||
|
||||
// TimesheetRow is the generated row type for table timesheet.
|
||||
type TimesheetRow struct {
|
||||
ID string
|
||||
CycleID string
|
||||
EmployeeID string
|
||||
Units int
|
||||
Approved bool
|
||||
ApprovedAt time.Time
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
}
|
||||
|
||||
// TimesheetCreateRequest is the generated create payload for table timesheet.
|
||||
type TimesheetCreateRequest struct {
|
||||
CycleID string `json:"cycleId"`
|
||||
EmployeeID string `json:"employeeId"`
|
||||
Units int `json:"units"`
|
||||
}
|
||||
|
||||
// CreateTimesheet inserts one timesheet row.
|
||||
func CreateTimesheet(ctx context.Context, db *sql.DB, req TimesheetCreateRequest) (TimesheetRow, error) {
|
||||
const q = `INSERT INTO timesheet (cycle_id, employee_id, units) VALUES ($1, $2, $3) RETURNING *`
|
||||
return scanTimesheet(db.QueryRowContext(ctx, q, req.CycleID, req.EmployeeID, req.Units))
|
||||
}
|
||||
|
||||
// GetTimesheet selects one timesheet row by primary key.
|
||||
func GetTimesheet(ctx context.Context, db *sql.DB, id string) (TimesheetRow, error) {
|
||||
const q = `SELECT * FROM timesheet WHERE id = $1`
|
||||
return scanTimesheet(db.QueryRowContext(ctx, q, id))
|
||||
}
|
||||
|
||||
// ApproveTimesheet flips the approved column on one timesheet row.
|
||||
func ApproveTimesheet(ctx context.Context, db *sql.DB, id string) (TimesheetRow, error) {
|
||||
const q = `UPDATE timesheet SET approved = true, approved_at = now() WHERE id = $1 RETURNING *`
|
||||
return scanTimesheet(db.QueryRowContext(ctx, q, id))
|
||||
}
|
||||
|
||||
// ListTimesheetsByCycle selects timesheet rows for one cycle.
|
||||
func ListTimesheetsByCycle(ctx context.Context, db *sql.DB, cycleID string) ([]TimesheetRow, error) {
|
||||
const q = `SELECT * FROM timesheet WHERE cycle_id = $1 ORDER BY employee_id`
|
||||
rows, err := db.QueryContext(ctx, q, cycleID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var out []TimesheetRow
|
||||
for rows.Next() {
|
||||
var r TimesheetRow
|
||||
if err := rows.Scan(&r.ID, &r.CycleID, &r.EmployeeID, &r.Units, &r.Approved,
|
||||
&r.ApprovedAt, &r.CreatedAt, &r.UpdatedAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, r)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
func scanTimesheet(row *sql.Row) (TimesheetRow, error) {
|
||||
var r TimesheetRow
|
||||
err := row.Scan(&r.ID, &r.CycleID, &r.EmployeeID, &r.Units, &r.Approved,
|
||||
&r.ApprovedAt, &r.CreatedAt, &r.UpdatedAt)
|
||||
return r, err
|
||||
}
|
||||
@@ -0,0 +1,139 @@
|
||||
// Code generated by protoc-gen-go. DO NOT EDIT.
|
||||
// versions:
|
||||
// protoc-gen-go v1.34.2
|
||||
// protoc v5.27.1
|
||||
// source: payroll/v1/payroll.proto
|
||||
|
||||
package payrollpb
|
||||
|
||||
import (
|
||||
protoreflect "google.golang.org/protobuf/reflect/protoreflect"
|
||||
protoimpl "google.golang.org/protobuf/runtime/protoimpl"
|
||||
timestamppb "google.golang.org/protobuf/types/known/timestamppb"
|
||||
)
|
||||
|
||||
// RunPayrollCycleRequest is the generated request message.
|
||||
type RunPayrollCycleRequest struct {
|
||||
state protoimpl.MessageState
|
||||
sizeCache protoimpl.SizeCache
|
||||
unknownFields protoimpl.UnknownFields
|
||||
|
||||
CycleId string `protobuf:"bytes,1,opt,name=cycle_id,json=cycleId,proto3" json:"cycle_id,omitempty"`
|
||||
DryRun bool `protobuf:"varint,2,opt,name=dry_run,json=dryRun,proto3" json:"dry_run,omitempty"`
|
||||
Reason string `protobuf:"bytes,3,opt,name=reason,proto3" json:"reason,omitempty"`
|
||||
}
|
||||
|
||||
func (x *RunPayrollCycleRequest) GetCycleId() string {
|
||||
if x != nil {
|
||||
return x.CycleId
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (x *RunPayrollCycleRequest) GetDryRun() bool {
|
||||
if x != nil {
|
||||
return x.DryRun
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (x *RunPayrollCycleRequest) GetReason() string {
|
||||
if x != nil {
|
||||
return x.Reason
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (x *RunPayrollCycleRequest) Reset() { *x = RunPayrollCycleRequest{} }
|
||||
func (x *RunPayrollCycleRequest) String() string { return protoimpl.X.MessageStringOf(x) }
|
||||
|
||||
// RunPayrollCycleResponse is the generated response message.
|
||||
type RunPayrollCycleResponse struct {
|
||||
state protoimpl.MessageState
|
||||
sizeCache protoimpl.SizeCache
|
||||
unknownFields protoimpl.UnknownFields
|
||||
|
||||
CycleId string `protobuf:"bytes,1,opt,name=cycle_id,json=cycleId,proto3" json:"cycle_id,omitempty"`
|
||||
Payslips []*Payslip `protobuf:"bytes,2,rep,name=payslips,proto3" json:"payslips,omitempty"`
|
||||
GrossCents int64 `protobuf:"varint,3,opt,name=gross_cents,json=grossCents,proto3" json:"gross_cents,omitempty"`
|
||||
NetCents int64 `protobuf:"varint,4,opt,name=net_cents,json=netCents,proto3" json:"net_cents,omitempty"`
|
||||
}
|
||||
|
||||
func (x *RunPayrollCycleResponse) GetPayslips() []*Payslip {
|
||||
if x != nil {
|
||||
return x.Payslips
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (x *RunPayrollCycleResponse) Reset() { *x = RunPayrollCycleResponse{} }
|
||||
func (x *RunPayrollCycleResponse) String() string { return protoimpl.X.MessageStringOf(x) }
|
||||
|
||||
// Payslip is the generated payslip message.
|
||||
type Payslip struct {
|
||||
state protoimpl.MessageState
|
||||
sizeCache protoimpl.SizeCache
|
||||
unknownFields protoimpl.UnknownFields
|
||||
|
||||
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
|
||||
CycleId string `protobuf:"bytes,2,opt,name=cycle_id,json=cycleId,proto3" json:"cycle_id,omitempty"`
|
||||
EmployeeId string `protobuf:"bytes,3,opt,name=employee_id,json=employeeId,proto3" json:"employee_id,omitempty"`
|
||||
GrossCents int64 `protobuf:"varint,4,opt,name=gross_cents,json=grossCents,proto3" json:"gross_cents,omitempty"`
|
||||
DeductionCents int64 `protobuf:"varint,5,opt,name=deduction_cents,json=deductionCents,proto3" json:"deduction_cents,omitempty"`
|
||||
NetCents int64 `protobuf:"varint,6,opt,name=net_cents,json=netCents,proto3" json:"net_cents,omitempty"`
|
||||
PeriodFrom *timestamppb.Timestamp `protobuf:"bytes,7,opt,name=period_from,json=periodFrom,proto3" json:"period_from,omitempty"`
|
||||
PeriodTo *timestamppb.Timestamp `protobuf:"bytes,8,opt,name=period_to,json=periodTo,proto3" json:"period_to,omitempty"`
|
||||
}
|
||||
|
||||
func (x *Payslip) GetId() string {
|
||||
if x != nil {
|
||||
return x.Id
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (x *Payslip) GetNetCents() int64 {
|
||||
if x != nil {
|
||||
return x.NetCents
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
func (x *Payslip) Reset() { *x = Payslip{} }
|
||||
func (x *Payslip) String() string { return protoimpl.X.MessageStringOf(x) }
|
||||
|
||||
// PayrollCycle is the generated cycle message.
|
||||
type PayrollCycle struct {
|
||||
state protoimpl.MessageState
|
||||
sizeCache protoimpl.SizeCache
|
||||
unknownFields protoimpl.UnknownFields
|
||||
|
||||
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
|
||||
Start *timestamppb.Timestamp `protobuf:"bytes,2,opt,name=start,proto3" json:"start,omitempty"`
|
||||
End *timestamppb.Timestamp `protobuf:"bytes,3,opt,name=end,proto3" json:"end,omitempty"`
|
||||
Status string `protobuf:"bytes,4,opt,name=status,proto3" json:"status,omitempty"`
|
||||
}
|
||||
|
||||
func (x *PayrollCycle) GetId() string {
|
||||
if x != nil {
|
||||
return x.Id
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (x *PayrollCycle) Reset() { *x = PayrollCycle{} }
|
||||
func (x *PayrollCycle) String() string { return protoimpl.X.MessageStringOf(x) }
|
||||
|
||||
var file_payroll_v1_payroll_proto_rawDesc = []byte{
|
||||
0x0a, 0x18, 0x70, 0x61, 0x79, 0x72, 0x6f, 0x6c, 0x6c, 0x2f, 0x76, 0x31, 0x2f, 0x70, 0x61, 0x79,
|
||||
0x72, 0x6f, 0x6c, 0x6c, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x0a, 0x70, 0x61, 0x79, 0x72,
|
||||
}
|
||||
|
||||
var file_payroll_v1_payroll_proto_goTypes = []any{
|
||||
(*RunPayrollCycleRequest)(nil),
|
||||
(*RunPayrollCycleResponse)(nil),
|
||||
(*Payslip)(nil),
|
||||
(*PayrollCycle)(nil),
|
||||
}
|
||||
|
||||
var File_payroll_v1_payroll_proto protoreflect.FileDescriptor
|
||||
@@ -0,0 +1,104 @@
|
||||
// Code generated by protoc-gen-go-grpc. DO NOT EDIT.
|
||||
// versions:
|
||||
// - protoc-gen-go-grpc v1.4.0
|
||||
// - protoc v5.27.1
|
||||
// source: payroll/v1/payroll.proto
|
||||
|
||||
package payrollpb
|
||||
|
||||
import (
|
||||
context "context"
|
||||
|
||||
grpc "google.golang.org/grpc"
|
||||
)
|
||||
|
||||
const (
|
||||
PayrollService_RunPayrollCycle_FullMethodName = "/payroll.v1.PayrollService/RunPayrollCycle"
|
||||
PayrollService_GetPayrollCycle_FullMethodName = "/payroll.v1.PayrollService/GetPayrollCycle"
|
||||
PayrollService_ListPayslips_FullMethodName = "/payroll.v1.PayrollService/ListPayslips"
|
||||
)
|
||||
|
||||
// PayrollServiceClient is the generated client API for PayrollService.
|
||||
type PayrollServiceClient interface {
|
||||
RunPayrollCycle(ctx context.Context, in *RunPayrollCycleRequest, opts ...grpc.CallOption) (*RunPayrollCycleResponse, error)
|
||||
GetPayrollCycle(ctx context.Context, in *RunPayrollCycleRequest, opts ...grpc.CallOption) (*PayrollCycle, error)
|
||||
ListPayslips(ctx context.Context, in *RunPayrollCycleRequest, opts ...grpc.CallOption) (*RunPayrollCycleResponse, error)
|
||||
}
|
||||
|
||||
type payrollServiceClient struct {
|
||||
cc grpc.ClientConnInterface
|
||||
}
|
||||
|
||||
// NewPayrollServiceClient returns a generated client.
|
||||
func NewPayrollServiceClient(cc grpc.ClientConnInterface) PayrollServiceClient {
|
||||
return &payrollServiceClient{cc}
|
||||
}
|
||||
|
||||
func (c *payrollServiceClient) RunPayrollCycle(ctx context.Context, in *RunPayrollCycleRequest, opts ...grpc.CallOption) (*RunPayrollCycleResponse, error) {
|
||||
out := new(RunPayrollCycleResponse)
|
||||
err := c.cc.Invoke(ctx, PayrollService_RunPayrollCycle_FullMethodName, in, out, opts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (c *payrollServiceClient) GetPayrollCycle(ctx context.Context, in *RunPayrollCycleRequest, opts ...grpc.CallOption) (*PayrollCycle, error) {
|
||||
out := new(PayrollCycle)
|
||||
err := c.cc.Invoke(ctx, PayrollService_GetPayrollCycle_FullMethodName, in, out, opts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (c *payrollServiceClient) ListPayslips(ctx context.Context, in *RunPayrollCycleRequest, opts ...grpc.CallOption) (*RunPayrollCycleResponse, error) {
|
||||
out := new(RunPayrollCycleResponse)
|
||||
err := c.cc.Invoke(ctx, PayrollService_ListPayslips_FullMethodName, in, out, opts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// PayrollServiceServer is the generated server API for PayrollService.
|
||||
type PayrollServiceServer interface {
|
||||
RunPayrollCycle(context.Context, *RunPayrollCycleRequest) (*RunPayrollCycleResponse, error)
|
||||
GetPayrollCycle(context.Context, *RunPayrollCycleRequest) (*PayrollCycle, error)
|
||||
ListPayslips(context.Context, *RunPayrollCycleRequest) (*RunPayrollCycleResponse, error)
|
||||
mustEmbedUnimplementedPayrollServiceServer()
|
||||
}
|
||||
|
||||
// UnimplementedPayrollServiceServer must be embedded for forward compatibility.
|
||||
type UnimplementedPayrollServiceServer struct{}
|
||||
|
||||
func (UnimplementedPayrollServiceServer) RunPayrollCycle(context.Context, *RunPayrollCycleRequest) (*RunPayrollCycleResponse, error) {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func (UnimplementedPayrollServiceServer) GetPayrollCycle(context.Context, *RunPayrollCycleRequest) (*PayrollCycle, error) {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func (UnimplementedPayrollServiceServer) ListPayslips(context.Context, *RunPayrollCycleRequest) (*RunPayrollCycleResponse, error) {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func (UnimplementedPayrollServiceServer) mustEmbedUnimplementedPayrollServiceServer() {}
|
||||
|
||||
// RegisterPayrollServiceServer registers the generated service.
|
||||
func RegisterPayrollServiceServer(s grpc.ServiceRegistrar, srv PayrollServiceServer) {
|
||||
s.RegisterService(&PayrollService_ServiceDesc, srv)
|
||||
}
|
||||
|
||||
// PayrollService_ServiceDesc is the generated service descriptor.
|
||||
var PayrollService_ServiceDesc = grpc.ServiceDesc{
|
||||
ServiceName: "payroll.v1.PayrollService",
|
||||
HandlerType: (*PayrollServiceServer)(nil),
|
||||
Methods: []grpc.MethodDesc{
|
||||
{MethodName: "RunPayrollCycle"},
|
||||
{MethodName: "GetPayrollCycle"},
|
||||
{MethodName: "ListPayslips"},
|
||||
},
|
||||
Metadata: "payroll/v1/payroll.proto",
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
package clock
|
||||
|
||||
import "time"
|
||||
|
||||
// Clock is the time seam so a payroll run is reproducible in tests.
|
||||
type Clock interface {
|
||||
Now() time.Time
|
||||
}
|
||||
|
||||
// System is the production clock.
|
||||
type System struct{}
|
||||
|
||||
func (System) Now() time.Time { return time.Now().UTC() }
|
||||
|
||||
// Fixed is a frozen clock.
|
||||
type Fixed struct{ At time.Time }
|
||||
|
||||
func (f Fixed) Now() time.Time { return f.At }
|
||||
@@ -0,0 +1,119 @@
|
||||
package payslipstore
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"sync"
|
||||
|
||||
"github.com/example/payroll-svc/internal/domain/payroll"
|
||||
)
|
||||
|
||||
// Store is the hand-written persistence seam the use-case layer writes through.
|
||||
// It is deliberately narrow: the generated fkit store can address every table,
|
||||
// this one only exposes the operations a payroll cycle needs.
|
||||
type Store struct {
|
||||
mu sync.RWMutex
|
||||
payslips map[string]payroll.Payslip
|
||||
cycles map[string]payroll.Cycle
|
||||
employees map[string][]payroll.Employee
|
||||
sheets map[string]payroll.Timesheet
|
||||
}
|
||||
|
||||
func New() *Store {
|
||||
return &Store{
|
||||
payslips: map[string]payroll.Payslip{},
|
||||
cycles: map[string]payroll.Cycle{},
|
||||
employees: map[string][]payroll.Employee{},
|
||||
sheets: map[string]payroll.Timesheet{},
|
||||
}
|
||||
}
|
||||
|
||||
func key(cycleID, employeeID string) string { return cycleID + "/" + employeeID }
|
||||
|
||||
// Upsert writes a payslip, replacing any prior slip for the same
|
||||
// (cycle, employee). A re-run of a cycle must not duplicate rows, so this is
|
||||
// an upsert rather than an insert.
|
||||
func (s *Store) Upsert(ctx context.Context, slip payroll.Payslip) error {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
if slip.CycleID == "" || slip.EmployeeID == "" {
|
||||
return fmt.Errorf("payslip missing cycle or employee id")
|
||||
}
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
s.payslips[key(slip.CycleID, slip.EmployeeID)] = slip
|
||||
return nil
|
||||
}
|
||||
|
||||
// ListByCycle returns every payslip a cycle produced.
|
||||
func (s *Store) ListByCycle(ctx context.Context, cycleID string) ([]payroll.Payslip, error) {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
out := make([]payroll.Payslip, 0, len(s.payslips))
|
||||
for _, slip := range s.payslips {
|
||||
if slip.CycleID == cycleID {
|
||||
out = append(out, slip)
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (s *Store) Cycle(ctx context.Context, cycleID string) (payroll.Cycle, error) {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return payroll.Cycle{}, err
|
||||
}
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
cycle, ok := s.cycles[cycleID]
|
||||
if !ok {
|
||||
return payroll.Cycle{}, fmt.Errorf("cycle %s not found", cycleID)
|
||||
}
|
||||
return cycle, nil
|
||||
}
|
||||
|
||||
func (s *Store) SaveCycle(ctx context.Context, cycle payroll.Cycle) error {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
s.cycles[cycle.ID] = cycle
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Store) EmployeesForCycle(ctx context.Context, cycleID string) ([]payroll.Employee, error) {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
return s.employees[cycleID], nil
|
||||
}
|
||||
|
||||
func (s *Store) Timesheet(ctx context.Context, cycleID, employeeID string) (payroll.Timesheet, error) {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return payroll.Timesheet{}, err
|
||||
}
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
ts, ok := s.sheets[key(cycleID, employeeID)]
|
||||
if !ok {
|
||||
return payroll.Timesheet{}, fmt.Errorf("no timesheet for %s in %s", employeeID, cycleID)
|
||||
}
|
||||
return ts, nil
|
||||
}
|
||||
|
||||
// Seed loads fixture data; the real service reads from Postgres.
|
||||
func (s *Store) Seed(cycle payroll.Cycle, employees []payroll.Employee, sheets []payroll.Timesheet) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
s.cycles[cycle.ID] = cycle
|
||||
s.employees[cycle.ID] = employees
|
||||
for _, ts := range sheets {
|
||||
s.sheets[key(ts.CycleID, ts.EmployeeID)] = ts
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"net/http"
|
||||
|
||||
"github.com/example/payroll-svc/internal/usecase/payroll"
|
||||
)
|
||||
|
||||
// PayrollHandler is the HTTP entry point into the payroll use-case layer.
|
||||
type PayrollHandler struct {
|
||||
svc *payroll.Service
|
||||
}
|
||||
|
||||
func NewPayrollHandler(svc *payroll.Service) *PayrollHandler {
|
||||
return &PayrollHandler{svc: svc}
|
||||
}
|
||||
|
||||
type runCycleRequest struct {
|
||||
DryRun bool `json:"dryRun"`
|
||||
Reason string `json:"reason"`
|
||||
}
|
||||
|
||||
type runCycleResponse struct {
|
||||
CycleID string `json:"cycleId"`
|
||||
Payslips int `json:"payslips"`
|
||||
GrossCents int64 `json:"grossCents"`
|
||||
NetCents int64 `json:"netCents"`
|
||||
}
|
||||
|
||||
// RunCycle kicks off a payroll cycle: it hands the cycle id to the use-case
|
||||
// layer, which builds and persists a payslip per active employee.
|
||||
func (h *PayrollHandler) RunCycle(w http.ResponseWriter, r *http.Request) {
|
||||
cycleID := r.PathValue("cycleID")
|
||||
if cycleID == "" {
|
||||
httpError(w, http.StatusBadRequest, "cycleID is required")
|
||||
return
|
||||
}
|
||||
|
||||
var req runCycleRequest
|
||||
if r.ContentLength > 0 {
|
||||
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
||||
httpError(w, http.StatusBadRequest, "malformed body")
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
result, err := h.svc.RunCycle(r.Context(), cycleID, payroll.RunOptions{
|
||||
DryRun: req.DryRun,
|
||||
Reason: req.Reason,
|
||||
})
|
||||
if err != nil {
|
||||
if errors.Is(err, payroll.ErrCycleClosed) {
|
||||
httpError(w, http.StatusConflict, "cycle already closed")
|
||||
return
|
||||
}
|
||||
httpError(w, http.StatusInternalServerError, "run failed")
|
||||
return
|
||||
}
|
||||
|
||||
writeJSON(w, http.StatusOK, runCycleResponse{
|
||||
CycleID: result.CycleID,
|
||||
Payslips: len(result.Payslips),
|
||||
GrossCents: result.TotalGrossCents,
|
||||
NetCents: result.TotalNetCents,
|
||||
})
|
||||
}
|
||||
|
||||
func (h *PayrollHandler) GetCycle(w http.ResponseWriter, r *http.Request) {
|
||||
cycle, err := h.svc.Cycle(r.Context(), r.PathValue("cycleID"))
|
||||
if err != nil {
|
||||
httpError(w, http.StatusNotFound, "no such cycle")
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, cycle)
|
||||
}
|
||||
|
||||
func (h *PayrollHandler) ListPayslips(w http.ResponseWriter, r *http.Request) {
|
||||
slips, err := h.svc.PayslipsForCycle(r.Context(), r.PathValue("cycleID"))
|
||||
if err != nil {
|
||||
httpError(w, http.StatusNotFound, "no such cycle")
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, slips)
|
||||
}
|
||||
|
||||
func writeJSON(w http.ResponseWriter, status int, body any) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(status)
|
||||
_ = json.NewEncoder(w).Encode(body)
|
||||
}
|
||||
|
||||
func httpError(w http.ResponseWriter, status int, msg string) {
|
||||
writeJSON(w, status, map[string]string{"error": msg})
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
package httpapi
|
||||
|
||||
import "net/http"
|
||||
|
||||
// NewRouter wires the HTTP surface. The payroll cycle endpoint is the only
|
||||
// entry point into the hand-written use-case layer.
|
||||
func NewRouter(h *PayrollHandler) http.Handler {
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("POST /v1/payroll/cycles/{cycleID}/run", h.RunCycle)
|
||||
mux.HandleFunc("GET /v1/payroll/cycles/{cycleID}", h.GetCycle)
|
||||
mux.HandleFunc("GET /v1/payroll/cycles/{cycleID}/payslips", h.ListPayslips)
|
||||
mux.HandleFunc("GET /healthz", health)
|
||||
return mux
|
||||
}
|
||||
|
||||
func health(w http.ResponseWriter, _ *http.Request) {
|
||||
w.WriteHeader(http.StatusOK)
|
||||
_, _ = w.Write([]byte("ok"))
|
||||
}
|
||||
@@ -0,0 +1,227 @@
|
||||
package payroll
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"sort"
|
||||
"time"
|
||||
|
||||
"github.com/example/payroll-svc/internal/domain/payroll"
|
||||
"github.com/example/payroll-svc/internal/platform/clock"
|
||||
"github.com/example/payroll-svc/internal/store/payslipstore"
|
||||
)
|
||||
|
||||
// ErrCycleClosed is returned when a cycle has already been finalized.
|
||||
var ErrCycleClosed = errors.New("payroll cycle is closed")
|
||||
|
||||
// ErrNoEmployees is returned when a cycle resolves to an empty roster.
|
||||
var ErrNoEmployees = errors.New("payroll cycle has no active employees")
|
||||
|
||||
// RunOptions tunes a single run of a payroll cycle.
|
||||
type RunOptions struct {
|
||||
// DryRun computes every payslip but persists nothing.
|
||||
DryRun bool
|
||||
// Reason is recorded on the audit trail for re-runs.
|
||||
Reason string
|
||||
// Only, when non-empty, restricts the run to these employee ids.
|
||||
Only []string
|
||||
}
|
||||
|
||||
// RunResult is the outcome of one payroll cycle run.
|
||||
type RunResult struct {
|
||||
CycleID string
|
||||
Payslips []payroll.Payslip
|
||||
TotalGrossCents int64
|
||||
TotalNetCents int64
|
||||
Skipped []string
|
||||
FinishedAt time.Time
|
||||
}
|
||||
|
||||
// Service is the hand-written payroll use-case layer. It owns the order of
|
||||
// operations for a cycle: resolve the roster, build a payslip per employee,
|
||||
// then persist. The generated CRUD layer under internal/gen has no opinion
|
||||
// about any of that — it can only read and write single rows.
|
||||
type Service struct {
|
||||
store *payslipstore.Store
|
||||
clock clock.Clock
|
||||
}
|
||||
|
||||
func NewService(store *payslipstore.Store, c clock.Clock) *Service {
|
||||
return &Service{store: store, clock: c}
|
||||
}
|
||||
|
||||
// RunCycle is the public entry point used by the HTTP handler. It loads the
|
||||
// cycle, guards its state, and delegates the actual work to runPayrollCycleAll.
|
||||
func (s *Service) RunCycle(ctx context.Context, cycleID string, opts RunOptions) (RunResult, error) {
|
||||
cycle, err := s.loadCycle(ctx, cycleID)
|
||||
if err != nil {
|
||||
return RunResult{}, err
|
||||
}
|
||||
if cycle.Status == payroll.CycleClosed {
|
||||
return RunResult{}, ErrCycleClosed
|
||||
}
|
||||
|
||||
roster, err := s.rosterFor(ctx, cycle, opts)
|
||||
if err != nil {
|
||||
return RunResult{}, err
|
||||
}
|
||||
if len(roster) == 0 {
|
||||
return RunResult{}, ErrNoEmployees
|
||||
}
|
||||
|
||||
return s.runPayrollCycleAll(ctx, cycle, roster, opts)
|
||||
}
|
||||
|
||||
// runPayrollCycleAll is the heart of the cycle: for every employee on the
|
||||
// roster it builds a payslip from that employee's contract and timesheet,
|
||||
// then upserts the result. Ordering matters — a payslip is only persisted
|
||||
// after every earning, deduction and tax line has been resolved, so a
|
||||
// partially-computed slip can never reach the store.
|
||||
func (s *Service) runPayrollCycleAll(
|
||||
ctx context.Context,
|
||||
cycle payroll.Cycle,
|
||||
roster []payroll.Employee,
|
||||
opts RunOptions,
|
||||
) (RunResult, error) {
|
||||
result := RunResult{CycleID: cycle.ID}
|
||||
now := s.clock.Now()
|
||||
|
||||
for _, employee := range roster {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return result, err
|
||||
}
|
||||
|
||||
timesheet, err := s.timesheetFor(ctx, cycle, employee)
|
||||
if err != nil {
|
||||
result.Skipped = append(result.Skipped, employee.ID)
|
||||
continue
|
||||
}
|
||||
|
||||
slip, err := s.BuildPayslip(ctx, cycle, employee, timesheet)
|
||||
if err != nil {
|
||||
return result, fmt.Errorf("build payslip for %s: %w", employee.ID, err)
|
||||
}
|
||||
|
||||
slip.RunAt = now
|
||||
slip.RunReason = opts.Reason
|
||||
|
||||
if !opts.DryRun {
|
||||
if err := s.store.Upsert(ctx, slip); err != nil {
|
||||
return result, fmt.Errorf("persist payslip for %s: %w", employee.ID, err)
|
||||
}
|
||||
}
|
||||
|
||||
result.Payslips = append(result.Payslips, slip)
|
||||
result.TotalGrossCents += slip.GrossCents
|
||||
result.TotalNetCents += slip.NetCents
|
||||
}
|
||||
|
||||
if !opts.DryRun {
|
||||
if err := s.closeCycle(ctx, cycle, now); err != nil {
|
||||
return result, err
|
||||
}
|
||||
}
|
||||
|
||||
sort.Slice(result.Payslips, func(i, j int) bool {
|
||||
return result.Payslips[i].EmployeeID < result.Payslips[j].EmployeeID
|
||||
})
|
||||
result.FinishedAt = now
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// rosterFor resolves which employees this cycle pays. An employee joins the
|
||||
// roster when their contract overlaps the cycle window and they are not on
|
||||
// unpaid leave for the whole period.
|
||||
func (s *Service) rosterFor(ctx context.Context, cycle payroll.Cycle, opts RunOptions) ([]payroll.Employee, error) {
|
||||
all, err := s.store.EmployeesForCycle(ctx, cycle.ID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
only := map[string]bool{}
|
||||
for _, id := range opts.Only {
|
||||
only[id] = true
|
||||
}
|
||||
|
||||
roster := make([]payroll.Employee, 0, len(all))
|
||||
for _, e := range all {
|
||||
if len(only) > 0 && !only[e.ID] {
|
||||
continue
|
||||
}
|
||||
if !e.Contract.OverlapsWindow(cycle.Start, cycle.End) {
|
||||
continue
|
||||
}
|
||||
if e.UnpaidLeaveCoversWindow(cycle.Start, cycle.End) {
|
||||
continue
|
||||
}
|
||||
roster = append(roster, e)
|
||||
}
|
||||
|
||||
sort.Slice(roster, func(i, j int) bool { return roster[i].ID < roster[j].ID })
|
||||
return roster, nil
|
||||
}
|
||||
|
||||
func (s *Service) timesheetFor(ctx context.Context, cycle payroll.Cycle, e payroll.Employee) (payroll.Timesheet, error) {
|
||||
ts, err := s.store.Timesheet(ctx, cycle.ID, e.ID)
|
||||
if err != nil {
|
||||
return payroll.Timesheet{}, err
|
||||
}
|
||||
if ts.Approved {
|
||||
return ts, nil
|
||||
}
|
||||
if e.Contract.Kind == payroll.ContractSalaried {
|
||||
// Salaried staff are paid the contractual period regardless of an
|
||||
// unapproved timesheet; hourly staff are skipped until approval.
|
||||
return payroll.Timesheet{
|
||||
CycleID: cycle.ID,
|
||||
EmployeeID: e.ID,
|
||||
Approved: true,
|
||||
Units: e.Contract.PeriodUnits(cycle.Start, cycle.End),
|
||||
}, nil
|
||||
}
|
||||
return payroll.Timesheet{}, fmt.Errorf("timesheet for %s not approved", e.ID)
|
||||
}
|
||||
|
||||
func (s *Service) loadCycle(ctx context.Context, cycleID string) (payroll.Cycle, error) {
|
||||
if cycleID == "" {
|
||||
return payroll.Cycle{}, errors.New("empty cycle id")
|
||||
}
|
||||
return s.store.Cycle(ctx, cycleID)
|
||||
}
|
||||
|
||||
func (s *Service) closeCycle(ctx context.Context, cycle payroll.Cycle, at time.Time) error {
|
||||
cycle.Status = payroll.CycleClosed
|
||||
cycle.ClosedAt = at
|
||||
return s.store.SaveCycle(ctx, cycle)
|
||||
}
|
||||
|
||||
// Cycle exposes a cycle for the read endpoints.
|
||||
func (s *Service) Cycle(ctx context.Context, cycleID string) (payroll.Cycle, error) {
|
||||
return s.loadCycle(ctx, cycleID)
|
||||
}
|
||||
|
||||
// PayslipsForCycle lists the payslips a completed cycle produced.
|
||||
func (s *Service) PayslipsForCycle(ctx context.Context, cycleID string) ([]payroll.Payslip, error) {
|
||||
slips, err := s.store.ListByCycle(ctx, cycleID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
sort.Slice(slips, func(i, j int) bool { return slips[i].EmployeeID < slips[j].EmployeeID })
|
||||
return slips, nil
|
||||
}
|
||||
|
||||
// Reopen unwinds a closed cycle so it can be re-run after a correction.
|
||||
func (s *Service) Reopen(ctx context.Context, cycleID string, reason string) error {
|
||||
cycle, err := s.loadCycle(ctx, cycleID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if cycle.Status != payroll.CycleClosed {
|
||||
return nil
|
||||
}
|
||||
cycle.Status = payroll.CycleOpen
|
||||
cycle.ReopenReason = reason
|
||||
cycle.ClosedAt = time.Time{}
|
||||
return s.store.SaveCycle(ctx, cycle)
|
||||
}
|
||||
@@ -0,0 +1,150 @@
|
||||
package payroll
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
|
||||
"github.com/example/payroll-svc/internal/domain/payroll"
|
||||
)
|
||||
|
||||
// BuildPayslip turns one employee's contract and timesheet into a complete
|
||||
// payslip for the cycle: base pay, overtime, allowances, then deductions and
|
||||
// tax, in that order. Every amount is in integer cents; nothing here rounds
|
||||
// until the final net, so a cent never disappears between two lines.
|
||||
//
|
||||
// This is the calculation the generated CRUD layer does NOT do — fkit's
|
||||
// BuildPayslip only copies fields between a DTO and a row.
|
||||
func (s *Service) BuildPayslip(
|
||||
ctx context.Context,
|
||||
cycle payroll.Cycle,
|
||||
employee payroll.Employee,
|
||||
timesheet payroll.Timesheet,
|
||||
) (payroll.Payslip, error) {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return payroll.Payslip{}, err
|
||||
}
|
||||
if timesheet.EmployeeID != "" && timesheet.EmployeeID != employee.ID {
|
||||
return payroll.Payslip{}, fmt.Errorf("timesheet/employee mismatch: %s vs %s", timesheet.EmployeeID, employee.ID)
|
||||
}
|
||||
|
||||
slip := payroll.Payslip{
|
||||
CycleID: cycle.ID,
|
||||
EmployeeID: employee.ID,
|
||||
Currency: employee.Contract.Currency,
|
||||
PeriodFrom: cycle.Start,
|
||||
PeriodTo: cycle.End,
|
||||
}
|
||||
|
||||
base := s.basePayCents(employee, cycle, timesheet)
|
||||
slip.Lines = append(slip.Lines, payroll.Line{
|
||||
Code: "BASE", Kind: payroll.LineEarning, AmountCents: base,
|
||||
})
|
||||
|
||||
if overtime := s.overtimeCents(employee, timesheet); overtime > 0 {
|
||||
slip.Lines = append(slip.Lines, payroll.Line{
|
||||
Code: "OT", Kind: payroll.LineEarning, AmountCents: overtime,
|
||||
})
|
||||
}
|
||||
|
||||
for _, allowance := range employee.Contract.Allowances {
|
||||
amount := prorateAllowance(allowance, cycle, employee)
|
||||
if amount == 0 {
|
||||
continue
|
||||
}
|
||||
slip.Lines = append(slip.Lines, payroll.Line{
|
||||
Code: allowance.Code, Kind: payroll.LineEarning, AmountCents: amount,
|
||||
})
|
||||
}
|
||||
|
||||
slip.GrossCents = sumKind(slip.Lines, payroll.LineEarning)
|
||||
|
||||
for _, d := range employee.Deductions {
|
||||
amount := d.AmountFor(slip.GrossCents)
|
||||
if amount == 0 {
|
||||
continue
|
||||
}
|
||||
slip.Lines = append(slip.Lines, payroll.Line{
|
||||
Code: d.Code, Kind: payroll.LineDeduction, AmountCents: amount,
|
||||
})
|
||||
}
|
||||
|
||||
tax, err := s.taxCents(employee, slip.GrossCents)
|
||||
if err != nil {
|
||||
return payroll.Payslip{}, fmt.Errorf("tax for %s: %w", employee.ID, err)
|
||||
}
|
||||
slip.Lines = append(slip.Lines, payroll.Line{
|
||||
Code: "TAX", Kind: payroll.LineDeduction, AmountCents: tax,
|
||||
})
|
||||
|
||||
slip.DeductionCents = sumKind(slip.Lines, payroll.LineDeduction)
|
||||
slip.NetCents = slip.GrossCents - slip.DeductionCents
|
||||
if slip.NetCents < 0 {
|
||||
slip.NetCents = 0
|
||||
slip.Underwater = true
|
||||
}
|
||||
|
||||
return slip, nil
|
||||
}
|
||||
|
||||
// basePayCents is the contractual pay for the period: salaried staff get the
|
||||
// period rate prorated across their contract window, hourly staff get rate ×
|
||||
// approved units.
|
||||
func (s *Service) basePayCents(e payroll.Employee, cycle payroll.Cycle, ts payroll.Timesheet) int64 {
|
||||
switch e.Contract.Kind {
|
||||
case payroll.ContractSalaried:
|
||||
full := e.Contract.PeriodRateCents
|
||||
return prorateSalary(full, e.Contract, cycle)
|
||||
case payroll.ContractHourly:
|
||||
return e.Contract.RateCents * int64(ts.Units)
|
||||
default:
|
||||
return 0
|
||||
}
|
||||
}
|
||||
|
||||
// overtimeCents pays approved units above the contractual threshold at the
|
||||
// contract's overtime multiplier.
|
||||
func (s *Service) overtimeCents(e payroll.Employee, ts payroll.Timesheet) int64 {
|
||||
if e.Contract.Kind != payroll.ContractHourly {
|
||||
return 0
|
||||
}
|
||||
threshold := e.Contract.OvertimeThresholdUnits
|
||||
if threshold <= 0 || ts.Units <= threshold {
|
||||
return 0
|
||||
}
|
||||
extra := int64(ts.Units - threshold)
|
||||
return int64(float64(e.Contract.RateCents) * e.Contract.OvertimeMultiplier * float64(extra))
|
||||
}
|
||||
|
||||
// taxCents applies the employee's tax band schedule to the gross.
|
||||
func (s *Service) taxCents(e payroll.Employee, gross int64) (int64, error) {
|
||||
if len(e.TaxBands) == 0 {
|
||||
return 0, nil
|
||||
}
|
||||
var tax int64
|
||||
remaining := gross
|
||||
for _, band := range e.TaxBands {
|
||||
if remaining <= 0 {
|
||||
break
|
||||
}
|
||||
if band.RateBasisPoints < 0 || band.RateBasisPoints > 10000 {
|
||||
return 0, fmt.Errorf("invalid band rate %d", band.RateBasisPoints)
|
||||
}
|
||||
slice := remaining
|
||||
if band.UpToCents > 0 && slice > band.UpToCents {
|
||||
slice = band.UpToCents
|
||||
}
|
||||
tax += slice * int64(band.RateBasisPoints) / 10000
|
||||
remaining -= slice
|
||||
}
|
||||
return tax, nil
|
||||
}
|
||||
|
||||
func sumKind(lines []payroll.Line, kind payroll.LineKind) int64 {
|
||||
var total int64
|
||||
for _, l := range lines {
|
||||
if l.Kind == kind {
|
||||
total += l.AmountCents
|
||||
}
|
||||
}
|
||||
return total
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
package payroll
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
"github.com/example/payroll-svc/internal/domain/payroll"
|
||||
)
|
||||
|
||||
// prorateSalary scales a full period rate down when the contract covers only
|
||||
// part of the cycle window (a mid-period joiner or leaver).
|
||||
func prorateSalary(fullCents int64, contract payroll.Contract, cycle payroll.Cycle) int64 {
|
||||
window := calendarDays(cycle.Start, cycle.End)
|
||||
if window <= 0 {
|
||||
return 0
|
||||
}
|
||||
covered := calendarDays(laterOf(cycle.Start, contract.StartsOn), earlierOf(cycle.End, contract.EndsOn))
|
||||
if covered >= window {
|
||||
return fullCents
|
||||
}
|
||||
if covered <= 0 {
|
||||
return 0
|
||||
}
|
||||
return fullCents * int64(covered) / int64(window)
|
||||
}
|
||||
|
||||
// prorateAllowance applies the same window rule to a recurring allowance.
|
||||
func prorateAllowance(a payroll.Allowance, cycle payroll.Cycle, e payroll.Employee) int64 {
|
||||
if !a.Prorated {
|
||||
return a.AmountCents
|
||||
}
|
||||
return prorateSalary(a.AmountCents, e.Contract, cycle)
|
||||
}
|
||||
|
||||
func calendarDays(from, to time.Time) int {
|
||||
if to.Before(from) {
|
||||
return 0
|
||||
}
|
||||
return int(to.Sub(from).Hours()/24) + 1
|
||||
}
|
||||
|
||||
func laterOf(a, b time.Time) time.Time {
|
||||
if b.IsZero() || a.After(b) {
|
||||
return a
|
||||
}
|
||||
return b
|
||||
}
|
||||
|
||||
func earlierOf(a, b time.Time) time.Time {
|
||||
if b.IsZero() || a.Before(b) {
|
||||
return a
|
||||
}
|
||||
return b
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"name": "starved-cluster-fixture",
|
||||
"private": true,
|
||||
"version": "0.0.0",
|
||||
"type": "module"
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
import { RequestChain, describeChain } from '../pipeline/chain';
|
||||
import type { PipelineRequest, PipelineResponse } from '../pipeline/types';
|
||||
import { openSocket } from '../transport/socket';
|
||||
|
||||
/**
|
||||
* The entry point a caller reaches for. Everything the chain does happens
|
||||
* underneath this call, which is why a flow question names it.
|
||||
*/
|
||||
export async function sendRequest(request: PipelineRequest): Promise<PipelineResponse> {
|
||||
const socket = openSocket(request.host, request.port);
|
||||
const chain = new RequestChain(request, socket);
|
||||
trace(describeChain(chain));
|
||||
return chain.proceed(request);
|
||||
}
|
||||
|
||||
export function trace(line: string): void {
|
||||
if (process.env.PIPELINE_TRACE) process.stderr.write(`${line}\n`);
|
||||
}
|
||||
|
||||
export async function sendAll(requests: PipelineRequest[]): Promise<PipelineResponse[]> {
|
||||
const out: PipelineResponse[] = [];
|
||||
for (const request of requests) out.push(await sendRequest(request));
|
||||
return out;
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
export interface ClientConfig {
|
||||
host: string;
|
||||
port: number;
|
||||
retries: number;
|
||||
userAgent: string;
|
||||
}
|
||||
|
||||
export function defaultConfig(): ClientConfig {
|
||||
return { host: 'localhost', port: 8080, retries: 3, userAgent: 'pipeline/1.0' };
|
||||
}
|
||||
|
||||
export function withHost(config: ClientConfig, host: string): ClientConfig {
|
||||
return { ...config, host };
|
||||
}
|
||||
@@ -0,0 +1,4 @@
|
||||
export { sendRequest, sendAll } from './app/client';
|
||||
export { RequestChain, describeChain } from './pipeline/chain';
|
||||
export { openSocket } from './transport/socket';
|
||||
export { defaultConfig } from './app/config';
|
||||
@@ -0,0 +1,318 @@
|
||||
import type { PipelineRequest, PipelineResponse, Interceptor, Socket } from './types';
|
||||
import { encodeFrame, decodeFrame } from './framing';
|
||||
import { defaultInterceptors } from './interceptors';
|
||||
|
||||
/**
|
||||
* A one-line summary of a chain, used only by the tracing hook in the caller.
|
||||
* It is TRIVIAL — it answers nothing about how a request travels — but it sits
|
||||
* next to the entry point in the call graph, so its cluster carries the file's
|
||||
* highest per-symbol importance.
|
||||
*/
|
||||
export function describeChain(chain: RequestChain): string {
|
||||
return `chain(${chain.index}/${chain.size}) -> ${chain.hostLabel}`;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
//
|
||||
// Everything below is the part a "how does a request reach the socket" question
|
||||
// is actually asking about. It is separated from the helper above by more than
|
||||
// the cluster gap threshold, so it forms its own cluster — a large one, whose
|
||||
// symbols are reached transitively rather than named.
|
||||
//
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export class RequestChain {
|
||||
readonly index: number;
|
||||
readonly size: number;
|
||||
readonly hostLabel: string;
|
||||
private readonly interceptors: Interceptor[];
|
||||
private readonly socket: Socket;
|
||||
private readonly request: PipelineRequest;
|
||||
private connectTimeoutMs = 10_000;
|
||||
private readTimeoutMs = 10_000;
|
||||
private writeTimeoutMs = 10_000;
|
||||
private calls = 0;
|
||||
|
||||
constructor(request: PipelineRequest, socket: Socket, index = 0, interceptors?: Interceptor[]) {
|
||||
this.request = request;
|
||||
this.socket = socket;
|
||||
this.index = index;
|
||||
this.interceptors = interceptors ?? defaultInterceptors();
|
||||
this.size = this.interceptors.length;
|
||||
this.hostLabel = `${request.host}:${request.port}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Run the request through the remaining interceptors and, once they are
|
||||
* exhausted, hand it to the transport. This is the method the flow question
|
||||
* is about: every hop between the caller and the socket passes through here.
|
||||
*/
|
||||
async proceed(request: PipelineRequest): Promise<PipelineResponse> {
|
||||
if (this.index >= this.size) {
|
||||
return this.writeAndRead(request);
|
||||
}
|
||||
this.calls += 1;
|
||||
if (this.calls > 1) {
|
||||
throw new Error(`chain link ${this.index} called ${this.calls} times`);
|
||||
}
|
||||
const next = this.advance(request);
|
||||
const interceptor = this.interceptors[this.index]!;
|
||||
const response = await interceptor.intercept(next);
|
||||
if (!response) {
|
||||
throw new Error(`interceptor ${interceptor.name} returned no response`);
|
||||
}
|
||||
if (this.index + 1 < this.size && next.callCount() === 0) {
|
||||
throw new Error(`interceptor ${interceptor.name} must call proceed()`);
|
||||
}
|
||||
return response;
|
||||
}
|
||||
|
||||
/**
|
||||
* The next link in the chain: the same chain with the cursor moved on and the
|
||||
* timeouts carried over. Cloning here is what keeps each interceptor from
|
||||
* mutating the chain the one before it is still holding.
|
||||
*/
|
||||
advance(request: PipelineRequest): RequestChain {
|
||||
const next = new RequestChain(request, this.socket, this.index + 1, this.interceptors);
|
||||
next.connectTimeoutMs = this.connectTimeoutMs;
|
||||
next.readTimeoutMs = this.readTimeoutMs;
|
||||
next.writeTimeoutMs = this.writeTimeoutMs;
|
||||
return next;
|
||||
}
|
||||
|
||||
callCount(): number {
|
||||
return this.calls;
|
||||
}
|
||||
|
||||
/**
|
||||
* The end of the chain: frame the request, put the bytes on the socket, wait
|
||||
* for the reply and decode it. Past this point there is no more pipeline —
|
||||
* this is the transport hop the question is looking for.
|
||||
*/
|
||||
private async writeAndRead(request: PipelineRequest): Promise<PipelineResponse> {
|
||||
const frame = encodeFrame(request);
|
||||
await this.socket.connect(this.connectTimeoutMs);
|
||||
await this.socket.write(frame, this.writeTimeoutMs);
|
||||
const raw = await this.socket.read(this.readTimeoutMs);
|
||||
const decoded = decodeFrame(raw);
|
||||
return {
|
||||
status: decoded.status,
|
||||
headers: decoded.headers,
|
||||
body: decoded.body,
|
||||
request,
|
||||
};
|
||||
}
|
||||
|
||||
withConnectTimeout(ms: number): RequestChain {
|
||||
const next = this.advance(this.request);
|
||||
next.connectTimeoutMs = checkDuration('connectTimeout', ms);
|
||||
return next;
|
||||
}
|
||||
|
||||
withReadTimeout(ms: number): RequestChain {
|
||||
const next = this.advance(this.request);
|
||||
next.readTimeoutMs = checkDuration('readTimeout', ms);
|
||||
return next;
|
||||
}
|
||||
|
||||
withWriteTimeout(ms: number): RequestChain {
|
||||
const next = this.advance(this.request);
|
||||
next.writeTimeoutMs = checkDuration('writeTimeout', ms);
|
||||
return next;
|
||||
}
|
||||
|
||||
connectTimeout(): number {
|
||||
return this.connectTimeoutMs;
|
||||
}
|
||||
|
||||
readTimeout(): number {
|
||||
return this.readTimeoutMs;
|
||||
}
|
||||
|
||||
writeTimeout(): number {
|
||||
return this.writeTimeoutMs;
|
||||
}
|
||||
|
||||
/**
|
||||
* Retry policy for the transport hop. Sits inside the same cluster as the
|
||||
* proceed/advance pair, so it is part of what a shrink has to choose between.
|
||||
*/
|
||||
async retryWrite(request: PipelineRequest, attempts: number): Promise<PipelineResponse> {
|
||||
let lastError: unknown;
|
||||
for (let attempt = 0; attempt < attempts; attempt += 1) {
|
||||
try {
|
||||
return await this.writeAndRead(request);
|
||||
} catch (error) {
|
||||
lastError = error;
|
||||
await backoff(attempt);
|
||||
}
|
||||
}
|
||||
throw lastError;
|
||||
}
|
||||
|
||||
/** Whether the chain may still be resumed after a transport failure. */
|
||||
canRetry(error: unknown): boolean {
|
||||
if (this.index >= this.size) return false;
|
||||
if (!(error instanceof Error)) return false;
|
||||
return error.message.includes('timeout') || error.message.includes('reset');
|
||||
}
|
||||
|
||||
/** The interceptor names, in the order the request will visit them. */
|
||||
route(): string[] {
|
||||
return this.interceptors.slice(this.index).map((i) => i.name);
|
||||
}
|
||||
|
||||
/** A copy of the chain rewound to the first interceptor. */
|
||||
rewind(): RequestChain {
|
||||
return new RequestChain(this.request, this.socket, 0, this.interceptors);
|
||||
}
|
||||
|
||||
/** Drop one interceptor by name and return the shortened chain. */
|
||||
without(name: string): RequestChain {
|
||||
const kept = this.interceptors.filter((i) => i.name !== name);
|
||||
return new RequestChain(this.request, this.socket, this.index, kept);
|
||||
}
|
||||
|
||||
/** Append an interceptor to the end of the chain. */
|
||||
with(interceptor: Interceptor): RequestChain {
|
||||
return new RequestChain(
|
||||
this.request,
|
||||
this.socket,
|
||||
this.index,
|
||||
[...this.interceptors, interceptor],
|
||||
);
|
||||
}
|
||||
|
||||
/** Close the transport this chain was built around. */
|
||||
async close(): Promise<void> {
|
||||
await this.socket.close();
|
||||
}
|
||||
|
||||
/** Headers the transport hop will actually put on the wire. */
|
||||
effectiveHeaders(): Record<string, string> {
|
||||
const headers: Record<string, string> = { ...this.request.headers };
|
||||
headers['host'] = this.hostLabel;
|
||||
headers['x-chain-index'] = String(this.index);
|
||||
headers['x-chain-size'] = String(this.size);
|
||||
if (this.request.body) headers['content-length'] = String(this.request.body.length);
|
||||
return headers;
|
||||
}
|
||||
|
||||
/** The request as the next link will see it, with the chain's headers merged. */
|
||||
prepared(): PipelineRequest {
|
||||
return { ...this.request, headers: this.effectiveHeaders() };
|
||||
}
|
||||
|
||||
/**
|
||||
* Send the prepared request through the rest of the chain. The convenience
|
||||
* wrapper most callers use instead of building the request themselves.
|
||||
*/
|
||||
async send(): Promise<PipelineResponse> {
|
||||
return this.proceed(this.prepared());
|
||||
}
|
||||
|
||||
/** Whether the chain has any interceptor left before the transport hop. */
|
||||
hasNext(): boolean {
|
||||
return this.index < this.size;
|
||||
}
|
||||
|
||||
/** The interceptor the next `proceed` will run, if there is one. */
|
||||
peek(): Interceptor | undefined {
|
||||
return this.interceptors[this.index];
|
||||
}
|
||||
|
||||
/** Total configured wait for one attempt, across all three timeouts. */
|
||||
totalTimeout(): number {
|
||||
return this.connectTimeoutMs + this.readTimeoutMs + this.writeTimeoutMs;
|
||||
}
|
||||
|
||||
/** Apply one timeout budget to all three phases at once. */
|
||||
withTimeout(ms: number): RequestChain {
|
||||
const next = this.advance(this.request);
|
||||
const checked = checkDuration('timeout', ms);
|
||||
next.connectTimeoutMs = checked;
|
||||
next.readTimeoutMs = checked;
|
||||
next.writeTimeoutMs = checked;
|
||||
return next;
|
||||
}
|
||||
|
||||
/**
|
||||
* Run the chain and translate a transport failure into a response, so a
|
||||
* caller that only cares about the status code never sees an exception.
|
||||
*/
|
||||
async sendOrStatus(status: number): Promise<PipelineResponse> {
|
||||
try {
|
||||
return await this.send();
|
||||
} catch {
|
||||
return {
|
||||
status,
|
||||
headers: this.effectiveHeaders(),
|
||||
body: new Uint8Array(),
|
||||
request: this.request,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/** A short description of where in the chain this link sits. */
|
||||
position(): string {
|
||||
return `${this.index + 1} of ${this.size + 1}`;
|
||||
}
|
||||
|
||||
/** The chain rebuilt around a different transport. */
|
||||
onSocket(socket: Socket): RequestChain {
|
||||
return new RequestChain(this.request, socket, this.index, this.interceptors);
|
||||
}
|
||||
|
||||
/**
|
||||
* Replay the request through the chain from the start, reusing the transport.
|
||||
* Used when an interceptor decides the response it got is not usable and the
|
||||
* whole pipeline has to run again against the same connection.
|
||||
*/
|
||||
async replay(): Promise<PipelineResponse> {
|
||||
const fresh = this.rewind();
|
||||
try {
|
||||
return await fresh.send();
|
||||
} finally {
|
||||
if (!fresh.hasNext()) await fresh.close();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate the chain before it runs: every interceptor named once, timeouts
|
||||
* inside their bounds, and a transport still open at the end of it.
|
||||
*/
|
||||
validate(): string[] {
|
||||
const problems: string[] = [];
|
||||
const seen = new Set<string>();
|
||||
for (const interceptor of this.interceptors) {
|
||||
if (seen.has(interceptor.name)) problems.push(`duplicate interceptor ${interceptor.name}`);
|
||||
seen.add(interceptor.name);
|
||||
}
|
||||
if (this.connectTimeoutMs <= 0) problems.push('connect timeout must be positive');
|
||||
if (this.readTimeoutMs <= 0) problems.push('read timeout must be positive');
|
||||
if (this.writeTimeoutMs <= 0) problems.push('write timeout must be positive');
|
||||
if (this.index > this.size) problems.push('chain cursor is past the end');
|
||||
return problems;
|
||||
}
|
||||
|
||||
/**
|
||||
* The transport hop on its own, with the chain's timeouts but none of its
|
||||
* interceptors — the escape hatch a caller uses to bypass the pipeline.
|
||||
*/
|
||||
async direct(request: PipelineRequest): Promise<PipelineResponse> {
|
||||
const problems = this.validate();
|
||||
if (problems.length > 0) throw new Error(problems.join('; '));
|
||||
return this.writeAndRead(request);
|
||||
}
|
||||
}
|
||||
|
||||
function checkDuration(name: string, ms: number): number {
|
||||
if (!Number.isFinite(ms) || ms < 0) throw new Error(`${name} must be a positive duration`);
|
||||
if (ms > 24 * 60 * 60 * 1000) throw new Error(`${name} is longer than a day`);
|
||||
return Math.round(ms);
|
||||
}
|
||||
|
||||
async function backoff(attempt: number): Promise<void> {
|
||||
const ms = Math.min(1000, 25 * 2 ** attempt);
|
||||
await new Promise((resolve) => setTimeout(resolve, ms));
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
import type { PipelineRequest } from './types';
|
||||
|
||||
export function encodeFrame(request: PipelineRequest): Uint8Array {
|
||||
const head = `${request.method} ${request.path}\n`;
|
||||
const headers = Object.entries(request.headers).map(([k, v]) => `${k}: ${v}`).join('\n');
|
||||
const text = `${head}${headers}\n\n`;
|
||||
const body = request.body ?? new Uint8Array();
|
||||
const out = new Uint8Array(text.length + body.length);
|
||||
out.set(new TextEncoder().encode(text), 0);
|
||||
out.set(body, text.length);
|
||||
return out;
|
||||
}
|
||||
|
||||
export function decodeFrame(raw: Uint8Array): { status: number; headers: Record<string, string>; body: Uint8Array } {
|
||||
const text = new TextDecoder().decode(raw);
|
||||
const split = text.indexOf('\n\n');
|
||||
const head = split < 0 ? text : text.slice(0, split);
|
||||
const lines = head.split('\n');
|
||||
const status = Number.parseInt(lines[0]?.split(' ')[1] ?? '0', 10);
|
||||
const headers: Record<string, string> = {};
|
||||
for (const line of lines.slice(1)) {
|
||||
const at = line.indexOf(': ');
|
||||
if (at > 0) headers[line.slice(0, at)] = line.slice(at + 2);
|
||||
}
|
||||
return { status, headers, body: raw.slice(split < 0 ? raw.length : split + 2) };
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
import type { Interceptor } from './types';
|
||||
|
||||
export function defaultInterceptors(): Interceptor[] {
|
||||
return [retryInterceptor(), headerInterceptor(), logInterceptor()];
|
||||
}
|
||||
|
||||
export function retryInterceptor(): Interceptor {
|
||||
return { name: 'retry', intercept: (chain) => chain.proceed(currentRequest()) };
|
||||
}
|
||||
|
||||
export function headerInterceptor(): Interceptor {
|
||||
return { name: 'headers', intercept: (chain) => chain.proceed(currentRequest()) };
|
||||
}
|
||||
|
||||
export function logInterceptor(): Interceptor {
|
||||
return { name: 'log', intercept: (chain) => chain.proceed(currentRequest()) };
|
||||
}
|
||||
|
||||
function currentRequest() {
|
||||
return { host: 'localhost', port: 80, method: 'GET', path: '/', headers: {} };
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
export interface PipelineRequest {
|
||||
host: string;
|
||||
port: number;
|
||||
method: string;
|
||||
path: string;
|
||||
headers: Record<string, string>;
|
||||
body?: Uint8Array;
|
||||
}
|
||||
|
||||
export interface PipelineResponse {
|
||||
status: number;
|
||||
headers: Record<string, string>;
|
||||
body: Uint8Array;
|
||||
request: PipelineRequest;
|
||||
}
|
||||
|
||||
export interface Interceptor {
|
||||
name: string;
|
||||
intercept(chain: { proceed(request: PipelineRequest): Promise<PipelineResponse> }): Promise<PipelineResponse>;
|
||||
}
|
||||
|
||||
export interface Socket {
|
||||
connect(timeoutMs: number): Promise<void>;
|
||||
write(frame: Uint8Array, timeoutMs: number): Promise<void>;
|
||||
read(timeoutMs: number): Promise<Uint8Array>;
|
||||
close(): Promise<void>;
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
import type { Socket } from '../pipeline/types';
|
||||
|
||||
/** Open a transport socket for a host/port pair. */
|
||||
export function openSocket(host: string, port: number): Socket {
|
||||
let open = false;
|
||||
const inbox: Uint8Array[] = [];
|
||||
return {
|
||||
async connect(timeoutMs: number) {
|
||||
if (open) return;
|
||||
await settle(timeoutMs);
|
||||
open = true;
|
||||
},
|
||||
async write(frame: Uint8Array, timeoutMs: number) {
|
||||
if (!open) throw new Error(`socket to ${host}:${port} is not connected`);
|
||||
await settle(timeoutMs);
|
||||
inbox.push(frame);
|
||||
},
|
||||
async read(timeoutMs: number) {
|
||||
await settle(timeoutMs);
|
||||
return inbox.shift() ?? new Uint8Array();
|
||||
},
|
||||
async close() {
|
||||
open = false;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async function settle(timeoutMs: number): Promise<void> {
|
||||
if (timeoutMs <= 0) throw new Error('timed out');
|
||||
}
|
||||
@@ -0,0 +1,39 @@
|
||||
# tail-render-ts — CG-38
|
||||
|
||||
An agent-named symbol sitting in the TAIL of a large file must render.
|
||||
|
||||
This mirrors the geometry of the reported file (a 1,414-line Svelte chat store)
|
||||
closely enough that the same two defects reproduce, and it is that geometry — not
|
||||
any individual line — that the fixture exists to hold:
|
||||
|
||||
| | line | why it matters |
|
||||
|---|---|---|
|
||||
| `QueuedMessage` (interface) | 70 | the DECOY. Same stem as the query token, near the top, cheap to render — it is what the broken build returned *instead of* the functions. |
|
||||
| `createSessionStore` (function) | 104–1417 | the ENVELOPE. Spans ~92% of the file, and `function` is deliberately **not** in `ENVELOPE_KINDS` (CG-27), so every symbol inside merges into ONE cluster that must then be shrunk and trimmed. |
|
||||
| `handleStreamMessage` | ~554 | a 290-line god-method in the middle, so the head of the file has plenty to spend the budget on. |
|
||||
| `queueMessage` | 1088 | TARGET. Past line 1,000. |
|
||||
| `removeQueuedMessage` | 1096 | TARGET. |
|
||||
| `flushQueuedMessages` | 1102 | TARGET. Past line 1,000. |
|
||||
|
||||
Two more pieces are load-bearing:
|
||||
|
||||
- **`queueMessage` never calls `flushQueuedMessages`** (both push to / drain the same
|
||||
array instead). That absence is what produced no call chain, no synthesized hop and
|
||||
no dispatch boundary — and so made `buildFlowFromNamedSymbols` throw the
|
||||
named-symbol identity away along with the narrative it had nothing to print.
|
||||
- **`types/worker-configuration.d.ts`** — 2,500 lines of generated Wrangler ambient
|
||||
types, carrying the `Generated by wrangler. DO NOT EDIT.` banner so the ranker flags
|
||||
and penalises it. It is what makes the fixture able to test the issue's
|
||||
index-dependence lead: a penalty on this file moves `maxGraph`, which moves the 6%
|
||||
relevance gate, which moves every other file's allowance — and must still not cost
|
||||
the top-ranked file the definitions the agent named.
|
||||
|
||||
`src/lib/session-store.ts` is machine-generated to hit those line numbers with real,
|
||||
extractable TypeScript. If you need to change it, change the geometry (the target
|
||||
line numbers, the closure span, the decoy's position) rather than editing individual
|
||||
lines — the fixture-shape assertions in
|
||||
`__tests__/explore-named-symbol-render.test.ts` will tell you if it has rotted.
|
||||
|
||||
Gate: `__tests__/explore-named-symbol-render.test.ts`.
|
||||
Probe: `node scripts/agent-eval/probe-named-symbol.mjs`.
|
||||
Numbers: `docs/benchmarks/explore-tail-render-cg38.md`.
|
||||
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"name": "tail-render-fixture",
|
||||
"private": true,
|
||||
"version": "0.0.0",
|
||||
"type": "module"
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
import { createSessionStore } from '../lib/session-store';
|
||||
|
||||
/** The composer owns the textarea and decides send-vs-queue. */
|
||||
export function createComposer(endpoint: string) {
|
||||
const store = createSessionStore({
|
||||
getProjectId: () => 'demo',
|
||||
getEndpoint: () => endpoint,
|
||||
onError: () => {},
|
||||
});
|
||||
let draft = '';
|
||||
|
||||
function setDraft(next: string) {
|
||||
draft = next;
|
||||
}
|
||||
|
||||
function submit(streaming: boolean) {
|
||||
if (streaming) store.queueMessage(draft);
|
||||
else store.sendMessage(draft, [], []);
|
||||
draft = '';
|
||||
}
|
||||
|
||||
function onTurnEnd() {
|
||||
store.flushQueuedMessages();
|
||||
}
|
||||
|
||||
return { setDraft, submit, onTurnEnd, store };
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user