Files
Matthew Kelch a5a0ddaeac feat(copilot): read per-request input/cache from session-store.db
The Copilot CLI and the GitHub Copilot desktop app both write
~/.copilot/session-store.db unconditionally; its assistant_usage_events
table holds one row per API request. Until now input/cache tokens for
these surfaces came only from the session.shutdown rollups in
events.jsonl, which are written only on clean shutdown (a crash loses
the whole leg's input/cache accounting) and lump each session leg into
one per-model total. The rollup also RESETS its counters at in-session
compaction (traced on a clean single-process 107-request session whose
sole rollup covered exactly its five post-compaction requests), so even
cleanly-closed long sessions were truncated; on a long-history machine
the store recovered ~35% of real Copilot spend lost to crashes and
compaction resets. The DB rows are per-request, crash-proof, and carry
real timestamps.

The store's input_tokens is cache-INCLUSIVE (input + cache_read +
cache_write), the same convention as the shutdown rollups — verified
against each row's token_details_json and by reconciling per-session
sums against the CLI's own footers and rollups across two machines
(1,380+ rows, 8 models, CLI 1.0.70–1.0.79, schema_version 6): every
divergence was a rollup gap. Emitted calls mirror the shutdown-call
contract: input/cache/reasoning only, output 0 — per-turn output stays
owned by the events.jsonl assistant.message calls.

Rollup-vs-store precedence is RECONCILED at serve time, per
(session, model), and only there. Both representations always parse and
cache; parseProviderSources aggregates the cached calls and, wherever
store rows exist for a (session, model), drops the rollup calls and
serves the rows plus per-leg RESIDUAL calls: each rollup leg subtracts
only the rows in its own interval — rows commit strictly before their
leg's shutdown line, so a leg at time T covers exactly the rows in
(previous leg's T, T] — and any remainder (per token component, floored
at zero) serves once at that leg's own timestamp. A store missing
requests a leg covered — adopted mid-session, rows pruned before ever
being read — therefore still serves that tail exactly once ON THAT
LEG'S DAY, a crash-tail row the rollup never saw can never cancel it,
and a complete store serves pure per-request granularity with every
residual retired to zero. The decision reads only cached contents, never discovery:
deleting or resetting the store changes nothing served, so finalized
daily history can never flip on an absence epoch; cached rows of a
deleted store remain the record until the 90-day orphan age-out (which
exempts still-discovered paths). The serve set is the one coherent
snapshot — nothing a writer does between discovery and a parse can
change what one pass sees — and read-time precedence heals persisted
duplication (stale epochs, runtimes without node:sqlite, restored
files) instead of preserving it, following the buildDurablePeriod
pattern.

Store rows and rollups carry supplementary accounting weight. A rollup
(or its residual) is aggregate accounting, never a request: zero
api-call/model-call/turn weight, tokens and cost fully retained. A
store row is one real request, but when it pairs with a served per-turn
call it is supplementary too; rows pair with same-model per-turn calls
by timestamp adjacency (monotone matching, tight 2-minute window — the
two are written at the same completion moment, and a wide window would
let a crash-only row pair against a neighbor whose own row is missing),
computed once over the FULL serve set so a date-range boundary that
separates a row from its call cannot double the request across adjacent
day queries. Only the unpaired rows — store-only requests, exactly
where crash-lost requests sit — count. Supplementary-only turns fold
into the nearest behavioral turn within 30 minutes; with no behavioral
turn to fold into they stay separate weightless turns, each on its own
day, with apiCalls 0 — and the session emission gate admits
usage-bearing zero-call sessions. The weight
propagates into the daily cache: aggregateProjectsIntoDays applies the
same rule to every calls counter and category-turn count it seals, so
v19 history and live summaries can never disagree about what was a
request.

A changed source whose read defers on the busy shape (locked, EACCES,
corrupt mid-replace — discovery still emits the source; only true
absence or a schema mismatch reads as absent) now marks session
hydration incomplete, so the daily backfill holds its watermark instead
of finalizing a day the deferred rows never reached; an unchanged
unreadable store defers nothing. The verdict travels with its result —
the 180s memo and the serve burst-reuse restore the hydration verdict
their cached data was parsed under, so a memoized partial parse cannot
inherit a later parse's complete — and a discovered source whose
FINGERPRINT cannot be read (EACCES on a present file) defers instead of
silently skipping, while a genuinely deleted file stays a silent skip. Copilot reasoning tokens are no longer
double-billed at the report layer: they are a subset of the output the
per-turn calls already price, and copilot joins claude in the
reasoning-inside-output case of the query-time cost recompute.

Store dedup keys are content-discriminated —
copilot-store:<sid>:<rowId>:<fnv1a64(created_at|tokens|model)> —
because AUTOINCREMENT prevents id reuse only within one database
lifetime: a same-path DB reset reusing row ids now mints new keys
instead of the durable union swallowing the new usage, while a
byte-identical re-insert still collapses (64-bit: 32-bit FNV
collisions between plausible token tuples are constructible). Every
call of a session serves under one project label resolved at serve
time — the session-state-derived label when the serve set knows it,
else the store rows' own — so neither rows cached before events.jsonl
existed nor an events.jsonl orphaned by a session-state prune can
split the session across two grouping keys.

CODEBURN_COPILOT_SESSION_STORE_DB is read but deliberately NOT
fingerprinted, per the #927 ruling (any copilot fingerprint change
drops cached entries whose path still exists, destroying pruned history
only the cache holds); the read is allowlisted in the #927 guard, and
serve-time reconciliation makes repointing safe without a fingerprint —
the new store's rows parse on sight and the old path's entries persist
as durable orphans. The copilot parse version appends session-store-v2
and the daily cache bumps v17 → v19: per-day attribution, call counts
and costs all change against pre-store builds. 19, not 18: an earlier
pushed head of this PR already claimed v18 under different accounting,
and the carry-forward would adopt those days as finalized without
re-deriving them.

Verified by A/B on snapshots of two real stores, a live SIGKILL crash
test (row present, no rollup, tokens recovered exactly), live resumes
whose warm-cache deltas matched new rows to the token, upgrade-healing
at 4,800-session scale, and serve-level regressions pinning every
maintainer finding from six review rounds: the rows-then-shutdown race,
stale-cache healing, age-out exemption, absence-epoch identity,
progressive row landing with residual retirement, behavioral weight
across all four pinned scenarios, the hydration fence, project
unification in both directions, the same-path reset, mixed
coverage (crash tail vs covered-leg gap), multi-leg residual day
attribution, range-invariant pairing, memo-scoped hydration verdicts,
and the fingerprint-failure fence.
2026-08-17 17:52:01 -04:00

979 lines
39 KiB
TypeScript

import { afterEach, beforeEach, describe, expect, it } from 'vitest'
import { readFile, rm, utimes, writeFile, mkdir } from 'fs/promises'
import { existsSync } from 'fs'
import { tmpdir } from 'os'
import { basename, join } from 'path'
import {
CACHE_VERSION,
PROVIDER_ENV_VARS,
type CachedCall,
type CachedFile,
type CachedTurn,
type FileFingerprint,
type SessionCache,
cleanupOrphanedTempFiles,
clearLoadCacheMemo,
computeEnvFingerprint,
emptyCache,
fingerprintFile,
loadCache,
mergeCallByDedupKey,
reconcileFile,
saveCache,
sessionCacheDir,
sourcePathStatCandidates,
} from '../src/session-cache.js'
import { readCacheOnDisk, writeCacheOnDisk } from './fixtures/session-cache-io.js'
// Version-suffixed directory (e.g. session-cache.v8) the cache now writes to.
const CACHE_DIR = () => basename(sessionCacheDir())
const TMP_DIR = join(tmpdir(), `codeburn-scache-test-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`)
beforeEach(() => {
process.env['CODEBURN_CACHE_DIR'] = TMP_DIR
})
afterEach(async () => {
if (existsSync(TMP_DIR)) await rm(TMP_DIR, { recursive: true })
})
function makeCall(overrides: Partial<CachedCall> = {}): CachedCall {
return {
provider: 'claude',
model: 'claude-sonnet-4-20250514',
usage: {
inputTokens: 1000,
outputTokens: 500,
cacheCreationInputTokens: 0,
cacheReadInputTokens: 0,
cachedInputTokens: 0,
reasoningTokens: 0,
webSearchRequests: 0,
cacheCreationOneHourTokens: 0,
},
speed: 'standard',
timestamp: '2026-05-15T10:00:00Z',
tools: ['Read', 'Edit'],
bashCommands: [],
skills: [],
deduplicationKey: 'msg-abc123',
...overrides,
}
}
function makeTurn(overrides: Partial<CachedTurn> = {}): CachedTurn {
return {
timestamp: '2026-05-15T10:00:00Z',
sessionId: 'sess-1',
userMessage: 'fix the bug',
calls: [makeCall()],
...overrides,
}
}
function makeCachedFile(overrides: Partial<CachedFile> = {}): CachedFile {
return {
fingerprint: { dev: 1, ino: 100, mtimeMs: 1000, sizeBytes: 5000 },
mcpInventory: [],
turns: [makeTurn()],
...overrides,
}
}
// ── emptyCache ─────────────────────────────────────────────────────────
describe('emptyCache', () => {
it('returns a valid empty cache', () => {
const cache = emptyCache()
expect(cache.version).toBe(CACHE_VERSION)
expect(cache.providers).toEqual({})
})
})
// ── loadCache / saveCache ──────────────────────────────────────────────
describe('loadCache / saveCache', () => {
it('returns empty cache when no file exists', async () => {
const cache = await loadCache()
expect(cache.version).toBe(CACHE_VERSION)
expect(cache.providers).toEqual({})
})
it('round-trips a cache through save and load', async () => {
const cache: SessionCache = {
version: CACHE_VERSION,
providers: {
claude: {
envFingerprint: 'abc123',
files: {
'/path/to/session.jsonl': makeCachedFile(),
},
},
},
}
await saveCache(cache)
const loaded = await loadCache()
expect(loaded).toEqual(cache)
})
it('persists a failed-parse marker across save/load (negative-result cache)', async () => {
const cache: SessionCache = {
version: CACHE_VERSION,
providers: {
pi: {
envFingerprint: 'abc123',
files: {
'/path/to/bad.jsonl': makeCachedFile({ turns: [], failed: true }),
},
},
},
}
await saveCache(cache)
const loaded = await loadCache()
// The `failed` flag and empty turns survive validation + load, so the file
// stays skipped on the next run instead of being re-read and re-thrown.
expect(loaded.providers['pi']?.files['/path/to/bad.jsonl']?.failed).toBe(true)
expect(loaded.providers['pi']?.files['/path/to/bad.jsonl']?.turns).toEqual([])
})
it('preserves the estimated-cost flag through save/load; a measured call stays unflagged', async () => {
const cache: SessionCache = {
version: CACHE_VERSION,
providers: {
warp: {
envFingerprint: 'abc123',
files: {
'/path/to/warp.sqlite': makeCachedFile({
turns: [makeTurn({ calls: [
makeCall({ deduplicationKey: 'est', costUSD: 0.5, isEstimated: true }),
makeCall({ deduplicationKey: 'measured', costUSD: 0.5 }),
] })],
}),
},
},
},
}
await saveCache(cache)
const loaded = await loadCache()
const calls = loaded.providers['warp']?.files['/path/to/warp.sqlite']?.turns[0]?.calls
expect(calls?.[0]?.isEstimated).toBe(true)
// A call with no flag round-trips as undefined, not silently coerced to true.
expect(calls?.[1]?.isEstimated).toBeUndefined()
})
it('returns empty cache on version mismatch', async () => {
const bad: SessionCache = { version: 999, providers: { claude: { envFingerprint: 'x', files: {} } } }
await mkdir(TMP_DIR, { recursive: true })
await writeFile(join(TMP_DIR, 'session-cache.json'), JSON.stringify(bad))
const loaded = await loadCache()
expect(loaded.version).toBe(CACHE_VERSION)
expect(loaded.providers).toEqual({})
})
it('returns empty cache on corrupt JSON', async () => {
await mkdir(TMP_DIR, { recursive: true })
await writeFile(join(TMP_DIR, 'session-cache.json'), '{broken')
const loaded = await loadCache()
expect(loaded.version).toBe(CACHE_VERSION)
expect(loaded.providers).toEqual({})
})
it('atomic write does not leave partial file on error', async () => {
await saveCache(emptyCache())
expect(await readCacheOnDisk()).toEqual(emptyCache())
})
})
// ── versioned filename + legacy adoption ───────────────────────────────
describe('versioned cache file + legacy adoption', () => {
function validCache(): SessionCache {
return {
version: CACHE_VERSION,
complete: false,
providers: { claude: { envFingerprint: 'abc123', files: { '/path/to/session.jsonl': makeCachedFile() } } },
}
}
it('writes and reads the version-suffixed directory, never the legacy name', async () => {
expect(basename(sessionCacheDir())).toBe(`session-cache.v${CACHE_VERSION}`)
await saveCache(validCache())
expect(existsSync(sessionCacheDir())).toBe(true)
expect(existsSync(join(TMP_DIR, 'session-cache.json'))).toBe(false)
expect(await loadCache()).toEqual(validCache())
})
it('adopts a matching-version legacy file once, without deleting or rewriting it', async () => {
await mkdir(TMP_DIR, { recursive: true })
const legacy = join(TMP_DIR, 'session-cache.json')
await writeFile(legacy, JSON.stringify(validCache()))
// Versioned directory absent → adopt-copy from legacy on first load.
expect(await loadCache()).toEqual(validCache())
expect(existsSync(sessionCacheDir())).toBe(true)
// Legacy left intact (not deleted, not rewritten).
expect(existsSync(legacy)).toBe(true)
expect(JSON.parse(await readFile(legacy, 'utf-8'))).toEqual(validCache())
// Adoption is one-time: a later legacy edit is ignored once the versioned
// file exists.
const mutated: SessionCache = { version: CACHE_VERSION, providers: { codex: { envFingerprint: 'zzz', files: {} } } }
await writeFile(legacy, JSON.stringify(mutated))
expect(await readCacheOnDisk()).toEqual(validCache())
})
it('ignores a different-version legacy file and never touches it', async () => {
await mkdir(TMP_DIR, { recursive: true })
const legacy = join(TMP_DIR, 'session-cache.json')
const stale = { version: 999, providers: { claude: { envFingerprint: 'x', files: {} } } }
await writeFile(legacy, JSON.stringify(stale))
expect((await loadCache()).providers).toEqual({})
// No versioned directory adopted; legacy left byte-intact.
expect(existsSync(sessionCacheDir())).toBe(false)
expect(JSON.parse(await readFile(legacy, 'utf-8'))).toEqual(stale)
})
it('saveCache never creates or overwrites a pre-existing legacy file', async () => {
await mkdir(TMP_DIR, { recursive: true })
const legacy = join(TMP_DIR, 'session-cache.json')
const legacyContent = JSON.stringify({ version: CACHE_VERSION, providers: {} })
await writeFile(legacy, legacyContent)
await saveCache(validCache())
// The shards hold the new data; the legacy file is byte-untouched.
expect(await readFile(legacy, 'utf-8')).toBe(legacyContent)
expect(await readCacheOnDisk()).toEqual(validCache())
})
})
// ── computeEnvFingerprint ──────────────────────────────────────────────
describe('computeEnvFingerprint', () => {
it('returns stable hash for same env', () => {
const a = computeEnvFingerprint('claude')
const b = computeEnvFingerprint('claude')
expect(a).toBe(b)
expect(a).toHaveLength(16)
})
it('changes when env var changes', () => {
const before = computeEnvFingerprint('claude')
process.env['CLAUDE_CONFIG_DIR'] = '/tmp/different'
const after = computeEnvFingerprint('claude')
expect(before).not.toBe(after)
})
it('returns stable hash for unknown provider (no env vars)', () => {
const a = computeEnvFingerprint('unknown-provider')
const b = computeEnvFingerprint('unknown-provider')
expect(a).toBe(b)
})
it('includes parser versions in provider fingerprints', () => {
expect(computeEnvFingerprint('claude')).not.toBe(computeEnvFingerprint('unknown-provider'))
expect(computeEnvFingerprint('copilot')).not.toBe(computeEnvFingerprint('unknown-provider'))
expect(computeEnvFingerprint('kiro')).not.toBe(computeEnvFingerprint('unknown-provider'))
expect(computeEnvFingerprint('warp')).not.toBe(computeEnvFingerprint('unknown-provider'))
})
})
// ── provider env overrides invalidate the fingerprint (#920) ─────────────
describe('provider env overrides invalidate the fingerprint (#920)', () => {
// Nine providers honored an env var that relocates where discovery looks
// without the var being declared in PROVIDER_ENV_VARS, so
// computeEnvFingerprint did not hash it and the cache section survived the
// change: sessions parsed from the old root kept being reported and the new
// root was never read. Each pair below must change the fingerprint when the
// var is set. codex/CODEX_HOME is the control — it already worked and must
// keep working.
const CASES: Array<[provider: string, varName: string]> = [
['kiro', 'KIRO_HOME'],
['grok', 'GROK_HOME'],
['kimi', 'KIMI_SHARE_DIR'],
['mux', 'MUX_ROOT'],
['mistral-vibe', 'VIBE_HOME'],
['zerostack', 'ZS_DATA_DIR'],
['codebuff', 'CODEBUFF_DATA_DIR'],
['goose', 'GOOSE_PATH_ROOT'],
['crush', 'CRUSH_GLOBAL_DATA'],
['codex', 'CODEX_HOME'],
]
const VARS = CASES.map(([, varName]) => varName)
// Save and restore every var we touch (beforeEach/afterEach), so a leaked
// env var never breaks unrelated tests in the same worker — and an ambient
// value never makes the "unset" case a lie.
const saved = new Map<string, string | undefined>()
beforeEach(() => {
for (const varName of VARS) {
saved.set(varName, process.env[varName])
delete process.env[varName]
}
})
afterEach(() => {
for (const varName of VARS) {
const original = saved.get(varName)
if (original === undefined) delete process.env[varName]
else process.env[varName] = original
}
})
for (const [provider, varName] of CASES) {
it(`changes the ${provider} fingerprint when ${varName} is set`, () => {
const unset = computeEnvFingerprint(provider)
process.env[varName] = '/tmp/codeburn-920-override'
const set = computeEnvFingerprint(provider)
expect(set).not.toBe(unset)
// Round trip: restoring the variable to its original state restores the
// original fingerprint, so the hash is a pure function of the
// environment.
delete process.env[varName]
expect(computeEnvFingerprint(provider)).toBe(unset)
})
}
it('changes the vercel-gateway fingerprint when AI_GATEWAY_API_KEY is set', () => {
const prev = process.env['AI_GATEWAY_API_KEY']
try {
const unset = computeEnvFingerprint('vercel-gateway')
process.env['AI_GATEWAY_API_KEY'] = 'sk-live-secret-abc'
const set = computeEnvFingerprint('vercel-gateway')
expect(set).not.toBe(unset)
delete process.env['AI_GATEWAY_API_KEY']
expect(computeEnvFingerprint('vercel-gateway')).toBe(unset)
} finally {
if (prev === undefined) delete process.env['AI_GATEWAY_API_KEY']
else process.env['AI_GATEWAY_API_KEY'] = prev
}
})
// Copilot is deliberately NOT declared in PROVIDER_ENV_VARS (Ruling 1 of
// lane 04): its OTel discovery returns one source per DB file
// ({ path: dbPath }, src/providers/copilot.ts:1935), and the durable
// carry-forward in getOrCreateProviderSection (src/parser.ts:2650) drops
// every cached entry whose source still exists on a fingerprint change — so
// declaring any CODEBURN_COPILOT_* var would force a re-parse that destroys
// conversations Copilot has since pruned from the DB, which only the cache
// still holds. The fingerprint must therefore NOT move when one is set.
// This reads as intent, not as an oversight — and the assertions below pin
// the WHOLE invariant (no entry at all, plus every one of the nine deferred
// reads), so a future "completing" edit fails a test instead of silently
// re-opening the durable history-loss path.
describe('copilot is deliberately undeclared in PROVIDER_ENV_VARS', () => {
it('has no PROVIDER_ENV_VARS entry at all', () => {
expect(PROVIDER_ENV_VARS['copilot']).toBeUndefined()
})
// The nine reads copilot.ts performs whose declaration is deferred (each
// is allowlisted in tests/provider-env-declarations.test.ts): setting any
// of them must leave the copilot fingerprint untouched.
const DEFERRED_COPILOT_VARS = [
'CODEBURN_COPILOT_SESSION_STATE_DIR',
'CODEBURN_COPILOT_OTEL_DB',
'CODEBURN_COPILOT_JETBRAINS_DIR',
'CODEBURN_COPILOT_WS_STORAGE_DIR',
'CODEBURN_COPILOT_GLOBAL_STORAGE_DIR',
'CODEBURN_COPILOT_DISABLE_OTEL',
'APPDATA',
'LOCALAPPDATA',
'XDG_CONFIG_HOME',
]
for (const varName of DEFERRED_COPILOT_VARS) {
it(`does not move the copilot fingerprint when ${varName} is set (deliberately undeclared)`, () => {
const prev = process.env[varName]
try {
const before = computeEnvFingerprint('copilot')
process.env[varName] = `/tmp/codeburn-copilot-920/${varName}`
expect(computeEnvFingerprint('copilot')).toBe(before)
delete process.env[varName]
expect(computeEnvFingerprint('copilot')).toBe(before)
} finally {
if (prev === undefined) delete process.env[varName]
else process.env[varName] = prev
}
})
}
})
})
// ── fingerprintFile ────────────────────────────────────────────────────
describe('fingerprintFile', () => {
it('returns fingerprint for existing file', async () => {
await mkdir(TMP_DIR, { recursive: true })
const filePath = join(TMP_DIR, 'test.jsonl')
await writeFile(filePath, 'line1\nline2\n')
const fp = await fingerprintFile(filePath)
expect(fp).not.toBeNull()
expect(fp!.sizeBytes).toBe(12)
expect(fp!.dev).toBeGreaterThan(0)
expect(fp!.ino).toBeGreaterThan(0)
expect(fp!.mtimeMs).toBeGreaterThan(0)
})
it('returns null for non-existent file', async () => {
const fp = await fingerprintFile('/no/such/file')
expect(fp).toBeNull()
})
it('resolves compound path with # separator (Cursor workspace)', async () => {
await mkdir(TMP_DIR, { recursive: true })
const filePath = join(TMP_DIR, 'state.vscdb')
await writeFile(filePath, 'cursor-data')
const fp = await fingerprintFile(`${filePath}#cursor-ws=__orphan__`)
expect(fp).not.toBeNull()
expect(fp!.sizeBytes).toBe(11)
})
it('resolves compound path with : separator (OpenCode session)', async () => {
await mkdir(TMP_DIR, { recursive: true })
const filePath = join(TMP_DIR, 'opencode.db')
await writeFile(filePath, 'opencode-data')
const fp = await fingerprintFile(`${filePath}:ses_abc123`)
expect(fp).not.toBeNull()
expect(fp!.sizeBytes).toBe(13)
})
it('returns null when base file does not exist for compound path', async () => {
const fp = await fingerprintFile('/no/such/file.db#cursor-ws=workspace')
expect(fp).toBeNull()
})
it('prefers # separator over : when both present', async () => {
await mkdir(TMP_DIR, { recursive: true })
const filePath = join(TMP_DIR, 'state.vscdb')
await writeFile(filePath, 'both-seps')
// Path has both # and : — should strip at # first and find the base file
const fp = await fingerprintFile(`${filePath}#cursor-ws=ws:extra-colon`)
expect(fp).not.toBeNull()
expect(fp!.sizeBytes).toBe(9)
})
// SQLite WAL mode parks committed writes in `<db>-wal`; the main file's
// stat only moves on checkpoint, which a long-lived writer defers for
// hours. A fingerprint from the main file alone reports data older than
// what is really committed (issue #913). The WAL sibling must be folded in.
it('folds -wal sibling into a # compound fingerprint (Hermes session)', async () => {
await mkdir(TMP_DIR, { recursive: true })
const dbPath = join(TMP_DIR, 'state.db')
await writeFile(dbPath, 'main-db')
const past = new Date(Date.now() - 48 * 3600 * 1000)
await utimes(dbPath, past, past)
await writeFile(`${dbPath}-wal`, 'wal-frames')
const fp = await fingerprintFile(`${dbPath}#hermes-session=abc`)
expect(fp).not.toBeNull()
// mtime: the fresh WAL wins over the checkpoint-stale main file.
expect(fp!.mtimeMs).toBeGreaterThan(past.getTime() + 3600 * 1000)
// size: main + wal, so WAL growth alone changes the fingerprint.
expect(fp!.sizeBytes).toBe('main-db'.length + 'wal-frames'.length)
})
it('folds -wal sibling into a : compound fingerprint (OpenCode session)', async () => {
await mkdir(TMP_DIR, { recursive: true })
const dbPath = join(TMP_DIR, 'opencode.db')
await writeFile(dbPath, 'oc-db')
const past = new Date(Date.now() - 48 * 3600 * 1000)
await utimes(dbPath, past, past)
await writeFile(`${dbPath}-wal`, 'oc-wal')
const fp = await fingerprintFile(`${dbPath}:ses_abc123`)
expect(fp).not.toBeNull()
expect(fp!.mtimeMs).toBeGreaterThan(past.getTime() + 3600 * 1000)
expect(fp!.sizeBytes).toBe('oc-db'.length + 'oc-wal'.length)
})
it('folds -wal sibling into a bare SQLite path (copilot agent-traces.db)', async () => {
await mkdir(TMP_DIR, { recursive: true })
const dbPath = join(TMP_DIR, 'agent-traces.db')
await writeFile(dbPath, 'traces')
const past = new Date(Date.now() - 48 * 3600 * 1000)
await utimes(dbPath, past, past)
await writeFile(`${dbPath}-wal`, 'traces-wal')
const fp = await fingerprintFile(dbPath)
expect(fp).not.toBeNull()
expect(fp!.mtimeMs).toBeGreaterThan(past.getTime() + 3600 * 1000)
expect(fp!.sizeBytes).toBe('traces'.length + 'traces-wal'.length)
})
it('keeps compound fingerprints working when no -wal sibling exists', async () => {
await mkdir(TMP_DIR, { recursive: true })
const dbPath = join(TMP_DIR, 'state.db')
await writeFile(dbPath, 'main-only')
const fp = await fingerprintFile(`${dbPath}#hermes-session=abc`)
expect(fp).not.toBeNull()
expect(fp!.sizeBytes).toBe('main-only'.length)
})
it('does not fold sibling files into non-SQLite fingerprints', async () => {
await mkdir(TMP_DIR, { recursive: true })
const filePath = join(TMP_DIR, 'session.jsonl')
await writeFile(filePath, 'jsonl-data')
// A stray neighbor that happens to match the -wal naming must not leak
// into a transcript fingerprint (offset-based append detection relies on
// sizeBytes being the transcript's real byte length).
await writeFile(`${filePath}-wal`, 'stray')
const fp = await fingerprintFile(filePath)
expect(fp).not.toBeNull()
expect(fp!.sizeBytes).toBe('jsonl-data'.length)
})
})
// ── reconcileFile ──────────────────────────────────────────────────────
describe('reconcileFile', () => {
it('returns "new" when no cached entry', () => {
const fp: FileFingerprint = { dev: 1, ino: 100, mtimeMs: 1000, sizeBytes: 5000 }
expect(reconcileFile(fp, undefined)).toEqual({ action: 'new' })
})
it('returns "unchanged" when all fields match', () => {
const fp: FileFingerprint = { dev: 1, ino: 100, mtimeMs: 1000, sizeBytes: 5000 }
const cached = makeCachedFile({ fingerprint: { ...fp } })
expect(reconcileFile(fp, cached)).toEqual({ action: 'unchanged' })
})
it('returns "appended" when ino same, size grew, and has lastCompleteLineOffset', () => {
const cached = makeCachedFile({
fingerprint: { dev: 1, ino: 100, mtimeMs: 1000, sizeBytes: 5000 },
lastCompleteLineOffset: 4500,
})
const current: FileFingerprint = { dev: 1, ino: 100, mtimeMs: 2000, sizeBytes: 8000 }
const result = reconcileFile(current, cached)
expect(result).toEqual({ action: 'appended', readFromOffset: 4500 })
})
it('returns "modified" when ino changed', () => {
const cached = makeCachedFile({
fingerprint: { dev: 1, ino: 100, mtimeMs: 1000, sizeBytes: 5000 },
})
const current: FileFingerprint = { dev: 1, ino: 200, mtimeMs: 2000, sizeBytes: 5000 }
expect(reconcileFile(current, cached)).toEqual({ action: 'modified' })
})
it('a failed marker at the same fingerprint stays "unchanged" (not re-parsed)', () => {
const fp: FileFingerprint = { dev: 1, ino: 100, mtimeMs: 1000, sizeBytes: 5000 }
const marker = makeCachedFile({ fingerprint: { ...fp }, turns: [], failed: true })
expect(reconcileFile(fp, marker)).toEqual({ action: 'unchanged' })
})
it('a failed marker is re-parsed once the file changes', () => {
const marker = makeCachedFile({
fingerprint: { dev: 1, ino: 100, mtimeMs: 1000, sizeBytes: 5000 },
turns: [],
failed: true,
})
const changed: FileFingerprint = { dev: 1, ino: 100, mtimeMs: 2000, sizeBytes: 6000 }
expect(reconcileFile(changed, marker)).toEqual({ action: 'modified' })
})
it('returns "modified" when the cached offset is stranded beyond the current EOF', () => {
// A truncate-then-regrow can leave the resume offset past live bytes; resuming
// there would drop the appended tail, so it must fall back to a full re-parse.
const cached = makeCachedFile({
fingerprint: { dev: 1, ino: 100, mtimeMs: 1000, sizeBytes: 5000 },
lastCompleteLineOffset: 9_000_000,
})
const current: FileFingerprint = { dev: 1, ino: 100, mtimeMs: 2000, sizeBytes: 8000 }
expect(reconcileFile(current, cached)).toEqual({ action: 'modified' })
})
it('returns "modified" when size shrank', () => {
const cached = makeCachedFile({
fingerprint: { dev: 1, ino: 100, mtimeMs: 1000, sizeBytes: 5000 },
lastCompleteLineOffset: 4500,
})
const current: FileFingerprint = { dev: 1, ino: 100, mtimeMs: 2000, sizeBytes: 3000 }
expect(reconcileFile(current, cached)).toEqual({ action: 'modified' })
})
it('returns "modified" when same size but different mtime', () => {
const cached = makeCachedFile({
fingerprint: { dev: 1, ino: 100, mtimeMs: 1000, sizeBytes: 5000 },
})
const current: FileFingerprint = { dev: 1, ino: 100, mtimeMs: 2000, sizeBytes: 5000 }
expect(reconcileFile(current, cached)).toEqual({ action: 'modified' })
})
it('returns "modified" for DB provider (no lastCompleteLineOffset) on any fingerprint change', () => {
const cached = makeCachedFile({
fingerprint: { dev: 1, ino: 100, mtimeMs: 1000, sizeBytes: 5000 },
})
const current: FileFingerprint = { dev: 1, ino: 100, mtimeMs: 2000, sizeBytes: 8000 }
expect(reconcileFile(current, cached)).toEqual({ action: 'modified' })
})
it('returns "modified" when dev changed even if ino same and size grew', () => {
const cached = makeCachedFile({
fingerprint: { dev: 1, ino: 100, mtimeMs: 1000, sizeBytes: 5000 },
lastCompleteLineOffset: 4500,
})
const current: FileFingerprint = { dev: 2, ino: 100, mtimeMs: 2000, sizeBytes: 8000 }
expect(reconcileFile(current, cached)).toEqual({ action: 'modified' })
})
})
// ── mergeCallByDedupKey ────────────────────────────────────────────────
describe('mergeCallByDedupKey', () => {
it('keeps earlier timestamp', () => {
const existing = makeCall({ timestamp: '2026-05-15T10:00:00Z' })
const incoming = makeCall({ timestamp: '2026-05-15T10:01:00Z' })
const merged = mergeCallByDedupKey(existing, incoming)
expect(merged.timestamp).toBe('2026-05-15T10:00:00Z')
})
it('takes incoming usage (latest wins)', () => {
const existing = makeCall({ usage: { ...makeCall().usage, outputTokens: 100 } })
const incoming = makeCall({ usage: { ...makeCall().usage, outputTokens: 999 } })
const merged = mergeCallByDedupKey(existing, incoming)
expect(merged.usage.outputTokens).toBe(999)
})
it('takes incoming tools (latest wins)', () => {
const existing = makeCall({ tools: ['Read'] })
const incoming = makeCall({ tools: ['Read', 'Edit', 'Bash'] })
const merged = mergeCallByDedupKey(existing, incoming)
expect(merged.tools).toEqual(['Read', 'Edit', 'Bash'])
})
})
// ── deep validation (loadCache) ────────────────────────────────────────
describe('loadCache validation', () => {
async function writeRawCache(data: unknown): Promise<void> {
await mkdir(TMP_DIR, { recursive: true })
await writeFile(join(TMP_DIR, 'session-cache.json'), JSON.stringify(data))
}
it('rejects providers as array', async () => {
await writeRawCache({ version: CACHE_VERSION, providers: [] })
expect((await loadCache()).providers).toEqual({})
})
it('rejects provider section missing envFingerprint', async () => {
await writeRawCache({ version: CACHE_VERSION, providers: { claude: { files: {} } } })
expect((await loadCache()).providers).toEqual({})
})
it('rejects provider section with files as array', async () => {
await writeRawCache({ version: CACHE_VERSION, providers: { claude: { envFingerprint: 'x', files: [] } } })
expect((await loadCache()).providers).toEqual({})
})
it('rejects file with invalid fingerprint (missing ino)', async () => {
await writeRawCache({
version: CACHE_VERSION,
providers: { claude: { envFingerprint: 'x', files: {
'/f': { fingerprint: { dev: 1, mtimeMs: 1, sizeBytes: 1 }, mcpInventory: [], turns: [] },
} } },
})
expect((await loadCache()).providers).toEqual({})
})
it('rejects file with non-numeric fingerprint field', async () => {
await writeRawCache({
version: CACHE_VERSION,
providers: { claude: { envFingerprint: 'x', files: {
'/f': { fingerprint: { dev: 1, ino: 'bad', mtimeMs: 1, sizeBytes: 1 }, mcpInventory: [], turns: [] },
} } },
})
expect((await loadCache()).providers).toEqual({})
})
it('rejects turn with missing sessionId', async () => {
const badTurn = { timestamp: 'x', userMessage: 'y', calls: [] }
await writeRawCache({
version: CACHE_VERSION,
providers: { claude: { envFingerprint: 'x', files: {
'/f': { fingerprint: { dev: 1, ino: 2, mtimeMs: 3, sizeBytes: 4 }, mcpInventory: [], turns: [badTurn] },
} } },
})
expect((await loadCache()).providers).toEqual({})
})
it('rejects call with missing usage object', async () => {
const badCall = { provider: 'claude', model: 'm', deduplicationKey: 'k', timestamp: 't', tools: [], bashCommands: [], skills: [] }
const turn = { timestamp: 'x', sessionId: 's', userMessage: 'y', calls: [badCall] }
await writeRawCache({
version: CACHE_VERSION,
providers: { claude: { envFingerprint: 'x', files: {
'/f': { fingerprint: { dev: 1, ino: 2, mtimeMs: 3, sizeBytes: 4 }, mcpInventory: [], turns: [turn] },
} } },
})
expect((await loadCache()).providers).toEqual({})
})
it('rejects call with NaN in usage', async () => {
const badUsage = { inputTokens: NaN, outputTokens: 0, cacheCreationInputTokens: 0, cacheReadInputTokens: 0, cachedInputTokens: 0, reasoningTokens: 0, webSearchRequests: 0, cacheCreationOneHourTokens: 0 }
const call = { provider: 'claude', model: 'm', usage: badUsage, deduplicationKey: 'k', timestamp: 't', tools: [], bashCommands: [], skills: [], speed: 'standard' }
const turn = { timestamp: 'x', sessionId: 's', userMessage: 'y', calls: [call] }
await writeRawCache({
version: CACHE_VERSION,
providers: { claude: { envFingerprint: 'x', files: {
'/f': { fingerprint: { dev: 1, ino: 2, mtimeMs: 3, sizeBytes: 4 }, mcpInventory: [], turns: [turn] },
} } },
})
expect((await loadCache()).providers).toEqual({})
})
function validCallJson() {
return {
provider: 'claude', model: 'm', deduplicationKey: 'k', timestamp: 't', speed: 'standard',
tools: ['Read'], bashCommands: ['ls'], skills: [],
usage: { inputTokens: 1, outputTokens: 1, cacheCreationInputTokens: 0, cacheReadInputTokens: 0, cachedInputTokens: 0, reasoningTokens: 0, webSearchRequests: 0, cacheCreationOneHourTokens: 0 },
}
}
function wrapCall(callOverride: Record<string, unknown>) {
return {
version: CACHE_VERSION,
providers: { claude: { envFingerprint: 'x', files: {
'/f': { fingerprint: { dev: 1, ino: 2, mtimeMs: 3, sizeBytes: 4 }, mcpInventory: [], turns: [
{ timestamp: 'x', sessionId: 's', userMessage: 'y', calls: [{ ...validCallJson(), ...callOverride }] },
] },
} } },
}
}
function wrapFile(fileOverride: Record<string, unknown>) {
return {
version: CACHE_VERSION,
providers: { claude: { envFingerprint: 'x', files: {
'/f': { fingerprint: { dev: 1, ino: 2, mtimeMs: 3, sizeBytes: 4 }, mcpInventory: [], turns: [], ...fileOverride },
} } },
}
}
it('rejects tools containing non-string element', async () => {
await writeRawCache(wrapCall({ tools: ['Read', 42] }))
expect((await loadCache()).providers).toEqual({})
})
it('rejects bashCommands containing object element', async () => {
await writeRawCache(wrapCall({ bashCommands: [{}] }))
expect((await loadCache()).providers).toEqual({})
})
it('rejects skills containing null element', async () => {
await writeRawCache(wrapCall({ skills: [null] }))
expect((await loadCache()).providers).toEqual({})
})
it('rejects invalid speed value', async () => {
await writeRawCache(wrapCall({ speed: 'turbo' }))
expect((await loadCache()).providers).toEqual({})
})
it('rejects non-string project', async () => {
await writeRawCache(wrapCall({ project: 123 }))
expect((await loadCache()).providers).toEqual({})
})
it('rejects non-string projectPath', async () => {
await writeRawCache(wrapCall({ projectPath: true }))
expect((await loadCache()).providers).toEqual({})
})
it('rejects mcpInventory containing non-string element', async () => {
await writeRawCache(wrapFile({ mcpInventory: ['valid', 99] }))
expect((await loadCache()).providers).toEqual({})
})
it('rejects non-numeric lastCompleteLineOffset', async () => {
await writeRawCache(wrapFile({ lastCompleteLineOffset: 'bad' }))
expect((await loadCache()).providers).toEqual({})
})
it('rejects NaN lastCompleteLineOffset', async () => {
await writeRawCache(wrapFile({ lastCompleteLineOffset: null }))
expect((await loadCache()).providers).toEqual({})
})
it('rejects non-string canonicalCwd', async () => {
await writeRawCache(wrapFile({ canonicalCwd: 42 }))
expect((await loadCache()).providers).toEqual({})
})
it('accepts optional fields when absent', async () => {
const cache: SessionCache = {
version: CACHE_VERSION,
providers: { claude: { envFingerprint: 'x', files: {
'/f': { fingerprint: { dev: 1, ino: 2, mtimeMs: 3, sizeBytes: 4 }, mcpInventory: [], turns: [] },
} } },
}
await writeRawCache(cache)
expect(await loadCache()).toEqual(cache)
})
it('accepts a fully valid cache with all fields populated', async () => {
const cache: SessionCache = {
version: CACHE_VERSION,
providers: {
claude: {
envFingerprint: 'abc',
files: { '/f': makeCachedFile() },
},
},
}
await writeRawCache(cache)
const loaded = await loadCache()
expect(loaded).toEqual(cache)
})
})
// ── cleanupOrphanedTempFiles ───────────────────────────────────────────
describe('cleanupOrphanedTempFiles', () => {
it('removes .tmp files older than 5 minutes', async () => {
await mkdir(TMP_DIR, { recursive: true })
await mkdir(join(TMP_DIR, CACHE_DIR()), { recursive: true })
const oldTmp = join(TMP_DIR, CACHE_DIR(), 'claude.abc123.json.tmp')
await writeFile(oldTmp, 'stale')
const { utimes } = await import('fs/promises')
const oldTime = new Date(Date.now() - 10 * 60 * 1000)
await utimes(oldTmp, oldTime, oldTime)
await cleanupOrphanedTempFiles()
expect(existsSync(oldTmp)).toBe(false)
})
it('preserves recent .tmp files', async () => {
await mkdir(TMP_DIR, { recursive: true })
await mkdir(join(TMP_DIR, CACHE_DIR()), { recursive: true })
const recentTmp = join(TMP_DIR, CACHE_DIR(), 'claude.def456.json.tmp')
await writeFile(recentTmp, 'recent')
await cleanupOrphanedTempFiles()
expect(existsSync(recentTmp)).toBe(true)
})
it('ignores .tmp files from other caches', async () => {
await mkdir(TMP_DIR, { recursive: true })
const otherTmp = join(TMP_DIR, 'codex-results.json.abc123.tmp')
await writeFile(otherTmp, 'other cache temp')
const { utimes } = await import('fs/promises')
const oldTime = new Date(Date.now() - 10 * 60 * 1000)
await utimes(otherTmp, oldTime, oldTime)
await cleanupOrphanedTempFiles()
expect(existsSync(otherTmp)).toBe(true)
})
it('does not fail when cache dir does not exist', async () => {
process.env['CODEBURN_CACHE_DIR'] = '/no/such/dir'
await cleanupOrphanedTempFiles()
})
})
// ── loadCache memo (serve fast-path) ─────────────────────────────────────
describe('loadCache memo', () => {
it('returns the identical object while the file is unchanged, reloads on rewrite', async () => {
clearLoadCacheMemo()
const cache = emptyCache()
cache.providers['memo-test'] = { envFingerprint: 'x', files: {} }
await saveCache(cache)
// saveCache write-through: the very object just published is served back.
const first = await loadCache()
expect(first).toBe(cache)
const second = await loadCache()
expect(second).toBe(first)
// An external rewrite (another process) moves mtime/size: fresh parse.
const external = emptyCache()
external.providers['memo-test-2'] = { envFingerprint: 'y', files: {} }
await saveCache(external)
const third = await loadCache()
expect(third).toBe(external)
expect(third).not.toBe(first)
clearLoadCacheMemo()
})
})
describe('sourcePathStatCandidates', () => {
it('mirrors the fingerprint fallbacks: plain, #-suffixed, and :-suffixed paths', () => {
expect(sourcePathStatCandidates('/a/b/state.vscdb')).toEqual(['/a/b/state.vscdb'])
expect(sourcePathStatCandidates('/a/b/state.vscdb#cursor-ws=ws1'))
.toEqual(['/a/b/state.vscdb#cursor-ws=ws1', '/a/b/state.vscdb'])
expect(sourcePathStatCandidates('/a/b/db.sqlite:sess-1'))
.toEqual(['/a/b/db.sqlite:sess-1', '/a/b/db.sqlite'])
// A plain Windows path must NOT yield the bare drive letter — a stat
// error on a cwd-relative 'C' must never hold hydration.
expect(sourcePathStatCandidates('C:\\data\\gone.jsonl')).toEqual(['C:\\data\\gone.jsonl'])
})
})
// A scoped load must read DURABLE_PROVIDER_NAMES providers in full even when
// the persisted section carries no durable stamp (a section written before
// the stamp landed, or by an older codeburn): copilot's serve-time
// reconciliation runs over the complete serve set, and a scope-skipped shard
// would silently make it range-dependent.
describe('scoped load durable-by-name exemption', () => {
it('loads an unstamped copilot section in full while scoping out a non-durable control', async () => {
const oldTurnTs = '2026-01-10T10:00:00.000Z'
const fileWithJanTurn = (): CachedFile => makeCachedFile({
turns: [{ ...makeTurn(), timestamp: oldTurnTs }],
})
await writeCacheOnDisk({
version: CACHE_VERSION,
complete: true,
providers: {
// Deliberately NO durable stamp on either section.
copilot: { envFingerprint: computeEnvFingerprint('copilot'), files: { '/x/session-store.db': fileWithJanTurn() } },
claude: { envFingerprint: computeEnvFingerprint('claude'), files: { '/x/old.jsonl': fileWithJanTurn() } },
},
})
const scoped = await loadCache({ fromMonth: '2026-06', toMonth: '2026-07' })
// The claude control proves the scope actually skipped the January shard;
// copilot's presence is then attributable only to the by-name exemption.
expect(scoped.providers['claude']?.files?.['/x/old.jsonl']).toBeUndefined()
expect(scoped.providers['copilot']?.files?.['/x/session-store.db']?.turns).toHaveLength(1)
})
})
// Capture-only copilot billing metadata must survive the sharded save/load
// round trip — including the per-shard call validation, which would silently
// drop the call if the optional-field checks and the fields ever disagree.
describe('copilot billing metadata round trip', () => {
it('keeps nanoAiu and requestMultiplier through save and load', async () => {
const file = makeCachedFile()
file.turns[0]!.calls[0] = { ...file.turns[0]!.calls[0]!, nanoAiu: 24594000000, requestMultiplier: 15 }
const cache: SessionCache = {
version: CACHE_VERSION,
complete: false,
providers: { copilot: { envFingerprint: 'fp-1', durable: true, files: { '/home/u/.copilot/session-store.db': file } } },
}
await saveCache(cache)
// Through the DISK, not the write-through memo — the read path is where
// the per-shard call validator could strip unknown or mistyped fields.
const loaded = await readCacheOnDisk()
const call = loaded.providers['copilot']!.files['/home/u/.copilot/session-store.db']!.turns[0]!.calls[0]!
expect(call.nanoAiu).toBe(24594000000)
expect(call.requestMultiplier).toBe(15)
})
})