Third R7b port, checklist-first recipe (docs/design/ruby-kernel-port-checklist.md). Grammar bump first, validated standalone (the rust pattern): tree-sitter-ruby ^0.20.1 (tree-sitter-wasms, 2024-02) → v0.23.1 — crate pinned =0.23.1, wasm built from tag 71bd32f's checked-in parser.c/scanner.c (both sha-matched against the crates.io tarball; content bump, ABI stays 14). Old-vs-new full-init dumps: sinatra/jekyll byte-identical; rails = exactly the one classified hunk (the `recv&.!=` safe-nav operator misparse fix, `table_name.!` → `table_name.!=`, precision-positive). Walker (python.rs chassis + the six ruby divergences) preserves bug-for-bug: the importTypes:['call'] funnel (class-body DSL — attr_accessor, has_many, define_method incl. its block, sinatra route blocks — emits NOTHING at non-body scope), hook-handled module multiply-capture (nested modules re-scan their subtree per level after popping — `this.hooked` fn-refs from class AND module AND file), the sibling-scan visibility trio (bare `private` invisible; `private :sym`/`private def` poison all later defs; the inner def stays public), bare-call statements (do…end body_statement emits, brace-block block_body doesn't), `.new` instantiates with last-`::`-segment names, constant-receiver references refs, require/require_relative path refs (posix-normalized, `.rb`-suffixed, `Kernel.require` and interpolated-path quirks included), `=begin` docstring marker survival, and the reverse-order value-ref DFS. Wire v2: the hook's mixin `implements` refs carry `filePath: ctx.filePath` — the ONE extraction-ref denormalized field (php's trait-use refs share the shape). RefRow's first pad byte becomes a flags slot (REF_FLAG_FILE_PATH); decode re-attaches its own filePath parameter; KERNEL_ABI_VERSION 1→2 on both sides (mismatched dist/.node pairs degrade to wasm, as designed). Gates: sweeps 0-diff sinatra 147/147, jekyll 164/164, rails 3452/3452 (3,763 files, 0 deferrals — ruby error incidence 0.00%, any deferral = walker bug); full-init dumps byte-identical ×3 (7.2k/9.4k/375.6k lines); kernel-ruby-parity suite (torture + CRLF + wire-flag pin + defer) + ruby grammar-parity row; full suite 2,613 green ×2 under CODEGRAPH_KERNEL_EXPECT=1 (one unrelated mcp-initialize timing flake under parallel load, passes solo 3/3 ×3). DEFAULT_ROUTED += ruby (12 langs). Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
49 KiB
Ruby kernel port (R7b) — the bug-for-bug checklist
Status: PORT COMPLETE (2026-07-20) — walker codegraph-kernel/src/ruby.rs,
all gates passed (grammar bump validated standalone — sinatra/jekyll dumps
byte-identical old-vs-new, rails exactly the one classified &.!= hunk;
parity sweeps 0-diff sinatra 147/147 / jekyll 164/164 / rails 3452/3452 with
0 deferrals; full-init dump gates byte-identical ×3; kernel-ruby-parity suite;
DEFAULT_ROUTED += ruby). One correction to this doc found during gating
(§Misc said "refs carry NO filePath"): the visitNode hook's mixin
implements refs DO carry filePath: ctx.filePath (languages/ruby.ts:45) —
the port added a v2 ref-flag wire slot (REF_FLAG_FILE_PATH, buffers.rs /
layout.ts, KERNEL_ABI_VERSION 1→2) and decode re-attaches its filePath
parameter; php's trait-use refs need the same bit. Survey basis:
every TS-side branch a .rb/.rake file exercises, with file:line anchors as
of f1ca991 (HEAD at survey time, clean main). Every grammar-shape claim below
was probed against both the old tree-sitter-wasms build and a fresh v0.23.1
build (probe scripts + outputs in the session scratchpad, svy-ruby/ — see
§Grammar prep), not assumed. Read WITH
docs/design/rust-kernel-migration-plan.md (§0a recipe, §5 gates) and the two
format precedents (rust-lang-kernel-port-checklist.md,
ccpp-kernel-port-checklist.md). Blocking findings: none — grammar bump is
shape-neutral on all extractor-relevant constructs (two classified deltas, one
inert, one precision-positive), error incidence 0.00% both arms on all three
gate repos, and python.rs is a close walker skeleton.
Grammar prep (do FIRST, land standalone — the rust recipe)
Ruby is NOT in VENDORED_WASM_LANGS (grammars.ts:291-304) — production loads
node_modules/tree-sitter-wasms/out/tree-sitter-ruby.wasm, built from
tree-sitter-ruby ^0.20.1 (tree-sitter-wasms 0.1.13 devDependency; 2024-02
era). The bump:
- Crate:
tree-sitter-ruby = "=0.23.1"(crates.io max, published 2024-11-11) = git tagv0.23.1= commit71bd32fb7607035768799732addba884a37a6210(also current master). sha256-verified crate-tarball ↔ tag, BOTH generated artifacts (ruby has an external scanner):src/parser.c4ce468358b6f4e25a35c8cf6bc0eaf60665bc22d602f8c939323c2347255cd15src/scanner.ce7a6196d6e78bf4c6728502e924c867dee5d851c6253e43fcdb8ba169009bc58
- ABI note (differs from the rust precedent): v0.23.1's checked-in
parser.c declares
LANGUAGE_VERSION 14— the tag predates the ABI-15 generator, so this is a grammar-CONTENT bump, ABI stays 14 (accepted by both web-tree-sitter and native tree-sitter 0.25, min-compatible 13). Don't expect an ABI change in kernel-grammar-parity; DO assert same-revision. - Build (from the CHECKED-IN parser.c — never run
tree-sitter generate):(brew emcc present; survey artifact sha256git clone https://github.com/tree-sitter/tree-sitter-ruby && cd tree-sitter-ruby git checkout v0.23.1 npx -y tree-sitter-cli@0.25.10 build --wasm -o tree-sitter-ruby.wasm .4cb5a4b12870876ca864c1e92fe1f5cd47036b2adc083e9306488af88867dbb4, 2,106,097 bytes, at scratchpadsvy-ruby/tree-sitter-ruby.wasm.) - Staging plan (grammar-bump PR, before any walker exists): vendor the
wasm to
src/extraction/wasm/tree-sitter-ruby.wasm; add'ruby'toVENDORED_WASM_LANGS(grammars.ts:291) with an R7b comment following the rust pattern (tag + sha-matched note); pintree-sitter-ruby = "=0.23.1"in codegraph-kernel/Cargo.toml under the exact-pin comment block (=like c/cpp/rust — crate + wasm move together or kernel-grammar-parity fails); kernel symboltree_sitter_ruby::LANGUAGEin langs.rs when the walker lands. Full suite green + the standalone bump gate (§Gates) before walker work starts. - Shape delta OLD→NEW — probed, complete classification (two deltas
total):
__END__data trailer — INERT. OLD has a separate__END__anon token + theuninterpretednode starting after the newline; NEW drops the__END__kind-table entries (353 → 351 kinds; kind ids renumber) anduninterpretedstarts right after__END__including the leading\nin its text. No extractor branch touchesuninterpreted(not in any ruby type list; recursed with nothing matching; file endLine comes fromsource.split('\n')), so no emission changes. Field table (32 fields) identical.- Safe-nav operator-method calls — BEHAVIOR-CHANGING,
precision-positive.
recv&.!= arg(railsactiverecord/lib/active_record/relation/where_clause.rb:62, the only hit in 3,763 real files): OLD misparses asassignment(recv&.!method!, then= arg) → wasm emittedrecv.!calls ref; NEW parses correctly ascallwithoperator-typed method!=→recv.!=calls ref, args walked as arguments. Wasm-path-only churn at bump time (both arms agree after); classify+accept in the bump gate's dump diff exactly like rust's "small precision-positive edge churn". Everything else probed byte-identical between OLD and NEW: full torture probe (modules/mixins/visibility/inline-def/calls/blocks/heredocs/hooks — 848-line CST dump identical), modern-syntax probe (endless methodsdef f(x) = …,.../&/*/**forwarding, case/in pattern matching, rightward assignment, hash shorthand, %-literals, operator method defs — identical incl. identical error behavior), CRLF variant (no errors either arm, identical shapes, node-type sequence == LF).
- Error incidence (both arms, all
.rb/.rake≤1MiB): sinatra 0/147 (0.00%/0.00%), jekyll 0/164, rails 0/3452 — zero disagreements. Ruby is a ts/java/py/go-class language: expect ~0% deferral, default--max-deferral 0.1with huge margin; double-digit deferral on a ruby sweep = broken walker (NO c/cpp 0.5 exemption). - Probe scripts + outputs live in the survey scratchpad
(
…/scratchpad/svy-ruby/):shape-probe-ruby.cjs(CST dumper, OLD vs NEW),table-compare.cjs(kind/field tables + error locator),error-sweep.cjs(per-repo has_error + full-CST sexp compare),extract-probe.cjs(runs the REAL dist extractor — itsextract-{dblcap,vis,req,vref,misc,reqedge}.txtdumps are the pinned ground truth cited throughout this doc, and double as walker test expectations),probe.rb/edge.rb/setter.rb/datatrailer.rb/probe-crlf.rb+ the extract fixtures,shape-{OLD,NEW}*.txtdumps,kinds-{OLD,NEW}.txt. Scratch dirs are throwaway — re-derive from this doc if gone.
Architecture decisions
- No preParse.
rubyExtractorhas nopreParsehook — the route point'spreParsedSource(kernel/index.ts:82) is a no-op; both arms parse raw bytes. Nothing to hoist. - Rails APPS take the DECODED path; the gate repos do NOT.
railsResolver(resolution/frameworks/ruby.ts:11,languages: ['ruby'], registered at frameworks/index.ts:52) has anextract()hook, and parse-worker.ts:93-99 forces any language with an applicable frameworkextract()onto the decodedextractFromSourcepath. Butdetect()(ruby.ts:22-39) needs a Gemfile containing'rails'(single-quoted!),config/application.rb,app/controllers/application_controller.rb, orconfig/routes.rbat repo root — none of sinatra/jekyll/rails-the-framework-repo trips it (verified), so all three parity repos exercise the raw buffer transport. Don't conclude the raw path is broken from a Rails-app perf run, and don't conclude decode is untested from the gate repos — the torture fixture suite covers decode via tests. - The framework extractor needs NO port — regex over raw source
(ruby.ts:109-190), runs identically after either arm inside
extractFromSource(tree-sitter.ts:6736-6758), mergingroutenodes +controller#actionrefs. §Frameworks pins its input contract. - One walker module (suggest
codegraph-kernel/src/rubylang.rsorruby.rs— no crate-language collision this time,ruby.rsis fine), registered in langs.rs (LANGUAGES+grammar_for+tree_sitter_ruby::LANGUAGE); per-filehas_error()→defer:like every walker.python.rsis the closest skeleton — shared shape: no braces, def-based, module-levelassignment→ always-variable(no isConst), fn-in-class-like → method, NOT a TYPE_ANNOTATION language, full value-ref machinery, walk from root with a scope stack. Ruby diverges from it in six places, each below: (a) thevisitNodehook (modules + mixins) runs FIRST for every node; (b)importTypes: ['call']swallows every top-level call; (c) a bespoke ruby branch in extractCall (receiver/method fields,.new→ instantiates, constant-receiver references); (d)extractBareCall(statement-level identifiers); (e) sibling-scan getVisibility; (f) fn-ref spec with empty idTypes +call/simple_symbolspecials. .rb/.rake→rubyat detectLanguage (grammars.ts:103-104), no content sniffing, no dialect. ExtensionlessGemfile/Rakefileresolve tounknown(lastIndexOf('.') === -1) and are never ruby. MAX_FILE_SIZE (1 MiB, extraction/index.ts:132),vendor/skip (Bundler, extraction/index.ts:168) and generated-file detection are orchestrator/TS-side and shared.method_call(see extractor config) does not exist in the grammar — probed: no such node kind in either build.
Extractor config (languages/ruby.ts — 147 lines, read it whole)
Types: functionTypes=[method]; classTypes=[class]; methodTypes=[method,
singleton_method]; interfaceTypes=[] (modules via the hook); structTypes=[];
enumTypes=[]; typeAliasTypes=[]; importTypes=[call] (the load-bearing
quirk — see visitNode dispatch); callTypes=[call, method_call — DEAD:
no such node kind in the grammar, keep it in the type-check for parity];
variableTypes=[assignment]. nameField=name, bodyField=body,
paramsField=parameters (unused — no getSignature). No enumMemberTypes,
propertyTypes, fieldTypes, packageTypes.
Hooks PRESENT (port each exactly):
- visitNode (ruby.ts:19-76) — runs for EVERY node before the dispatch
ladder (tree-sitter.ts:943-953). Two jobs:
- Mixins:
callwith NOreceiverfield whosemethodfield text isinclude/extend/prepend→ for each namedChild of theargumentsfield (?? namedChildren.find(type==='argument_list')) of typeconstant|scope_resolution, push animplementsunresolved ref {fromNodeId: nodeStack top, referenceName: the arg's FULL text (Foo::Barverbatim), line/column: the CALL node's start (same line+col for every arg of one call)} — then return true (handled; the call never reaches the ladder). Gates that must hold: nodeStack non-empty AND an args node found, else fall through unhandled.extend self→ arg typeself, skipped (no ref) but still handled. Receiver formFoo.include Barhas a receiver → hook declines → the call dies in extractImport (nothing emitted). - Modules: node type
modulewith anamefield → create amodulenode (name = name-field TEXT —A::Bverbatim formodule A::B; no docstring, no visibility, no extras — ctx.createNode with no extra), push its id, visit each namedChild of thebodyfield via ctx.visitNode, pop, return true. No name field → false (fall through: children visited bare; can't happen in valid ruby). When the hook handles a node, the dispatcher runsscanFnRefSubtree(tree-sitter.ts:951) on it — capture-only, halts at nested functionTypes (method) at depth>0 but NOT at class/module nodes, depth ≤12. QUIRK, EMPIRICALLY PINNED (dist probeextract-dblcap.txtin the survey scratchpad): hook-handled modules MULTIPLY-CAPTURE fn-ref containers. The hook's inner ctx.visitNode walk captures once at the true scope; then the post-hook scan re-captures the SAME containers with fromNodeId = the stack top at scan time (the module already popped) — and nested modules compound (each level's hook return triggers another scan of its whole subtree). Ground truth formodule A { module B { class C < Base { before_action :hooked … } } }: THREEfunction_ref "this.hooked"refs — from class:C (inner walk), from module:A (scan of B, run while A still pushed), from file (scan of A) — in exactly that candidate order, all flushed (this.-prefixed candidates skip the gate; flush dedupe is per (fromNodeId,name) so distinct scopes all survive). Class-body containers NOT inside any module capture once (a bareclass Cat file scope isn't hook-handled). Reproduce the multiplication exactly — Rails (module Admin; class XController; before_action …) hits it everywhere. Mixin-handled calls are also scanned (their argument_list yields nothing — constants/self aren't candidate shapes).
- Mixins:
- extractBareCall (ruby.ts:77-105) — called ONLY from visitFunctionBody
(tree-sitter.ts:5159-5173), for nodes that aren't callTypes/instantiation:
node type must be
identifier; parent type must be in BLOCK_PARENTS = {body_statement,then,else,do,begin,rescue,ensure,when}; name not in SKIP = {true,false,nil,self,super,FILE,LINE, dir}; first char NOT ASCII A-Z (charCodeAt(0)in [65,90] — ASCII only: a Unicode-uppercase identifier is NOT skipped); else return the name → the dispatcher emits onecallsref {caller = stack top, name, line = identifier startRow+1, col = startColumn}. Consequences (all probed): statement-levelresetin a def/do_block body → ref; brace-block bodies areblock_body— NOT in the set →5.times { beep }emits NOTHING for beep whiledo…endbodies (body_statement) do; modifier forms (cleanup unless done?,compute rescue nil) have parentsunless_modifier/rescue_modifier→ NOTHING for the body identifier; ternary branches (parentconditional) → nothing; bare identifiers insideinterpolation→ nothing;begin/rescue(via itsthenbody)/else/ensure/when(then)/while(bodydo) statement identifiers → refs. NOTE?/!-suffixed zero-arg invocations (done?,block_given?) parse ascall(method field only) — they take extractCall, not this hook, so they get refs in ANY position including conditions. - getVisibility (ruby.ts:106-122) — walk
previousNamedSiblingchain from the def node (unbounded, through non-matching siblings); first sibling of typecallwhosemethodfield text isprivate/protected/publicdecides; else'public'. Probed quirks, all PRESERVE:- A bare
private/protected/publicline parses asidentifier, notcall→ invisible — methods after a bare modifier stay 'public'. private :greet/private def x; endARE calls with methodprivate→ every def AFTER them (any distance) gets 'private', regardless of ruby's actual semantics (arg-scoped forprivate :sym).- The def INSIDE
private def foosits in the call's argument_list with no previous named siblings → 'public'. - Applies to
extractFunction,extractMethod,extractClass(a class after aprivate :xcall gets 'private'), and top-level defs (scan runs at program level). extractInterface/module hook don't compute visibility.
- A bare
- extractImport (ruby.ts:123-146) — called for every
callreaching the ladder's importTypes branch. signature =source.substring(startIndex, endIndex).trim()(UTF-16 substring) of the whole call. Gate: FIRST namedChild of typeidentifier(this is normally themethodfield child; for a receiver callfoo.requireit finds the receiverfoofirst and declines; forKernel.require "x"the receiver is aconstantso the find reaches the METHOD identifierrequire→ treated as a require — preserve) with text exactlyrequire|require_relative; else null. Then: first namedChild of typeargument_list→ its first namedChild of typestring→ its first namedChild of typestring_content→ moduleName = content text; any miss → null (require :sym→ nothing). PROBED:%q()IS astringnode →require %q(pct/lib)is a full require (import node- refs); an INTERPOLATED path takes the FIRST string_content only —
require "interp/#{x}"→ moduleNameinterp/→ refsinterp/+interp/.rb(garbage but deterministic — preserve). Returns {moduleName, signature}, no handledRefs → the generic path also fires (see extractImport below).
- refs); an INTERPOLATED path takes the FIRST string_content only —
Hooks ABSENT (the walker must NOT do these): preParse, resolveName,
recoverMangledName, isMisparsedFunction, isConst, isStatic,
isExported, isAsync, getSignature, getReturnType, getReceiverType,
resolveBody, classifyClassNode, classifyMethodNode,
extractPropertyName, interfaceKind, extraClassNodeTypes,
packageTypes/extractPackage, extractModifiers, synthesizeMembers,
skipBodilessClass, methodsAreTopLevel. Consequences: every ruby
function/method node has signature/isAsync/isStatic/returnType/isExported
undefined (file node isExported:false); no isConst means every
assignment extracts kind 'variable', never 'constant' — including
MAX = 3 (value-ref targeting still works: kind variable is a target);
attr_accessor/attr_reader/attr_writer synthesize NOTHING (no
synthesizeMembers — they're plain class-body calls that emit nothing at all);
no parameter nodes, no decorates refs (extractDecoratorsFor
tree-sitter.ts:4897 runs but ruby has no decorator/annotation/
marker_annotation node kinds and the backward scan stops at the first
non-decorator sibling — always a no-op).
tree-sitter.ts branches (anchors as of f1ca991)
visitNode dispatch — what each ruby node hits (ladder at 936-1303)
| Node | Branch | Behavior |
|---|---|---|
| every node | visitNode hook first (943) | mixin calls + modules handled there (above); handled → scanFnRefSubtree + STOP |
| every node | maybeCaptureFnRefs (990) | fires for argument_list / pair (the RUBY_SPEC dispatch keys) in visitNode context too — this is how class-body hook-DSL symbols are captured |
method |
functionTypes:994 | inside class-like (class/module/…, isInsideClassLikeNode:1486 — module counts, 1498) AND in methodTypes → extractMethod:1737; else extractFunction:1517. skipChildren |
singleton_method (def self.x / def Foo.x / def obj.x) |
NOT functionTypes → methodTypes:1027 | no classifyMethodNode → extractMethod. At top level: not class-like, no methodsAreTopLevel, no receiver hook → gate 1747 sends it to extractFunction → a top-level def self.x is a plain function node named x. object field ignored everywhere — def self.x vs def x are indistinguishable in the graph (isStatic undefined) |
class |
classTypes:1005 | no classifyClassNode → extractClass:1679 (kind 'class') |
singleton_class (class << self) |
NO branch | recursed → its body body_statement's defs extract as methods of the OUTER class, indistinguishable from instance methods (probed) |
assignment (top level / class / module) |
variableTypes:1098 | gate: !isInsideClassLikeNode() || isClassScopeConstantAssignment (1508: type==='assignment' AND (left field ?? namedChild(0)).type === 'constant'). File scope: identifier AND constant LHS both extract. Class/module scope: ONLY constant-LHS. Then extractVariable:2538 + scanFnRefSubtree (1110) + skipChildren — the RHS is never walked: top-level/class-level X = Foo.new emits NO instantiates/calls |
operator_assignment (+=, ||=) |
no branch | not assignment → nothing extracted at any scope; recursed (children emit nothing either at non-body scope) |
call (top level / class / module body) |
importTypes:1209 — NEVER callTypes | extractImport:3170 (below). require/require_relative → import node + refs; every other call → NOTHING (hook null → 3350 if (this.extractor.extractImport) return;). skipChildren stays false → children ARE visited: nested calls also land here (nothing), argument_lists get fn-ref capture. This kills attr_accessor, has_many, define_method (+ its do_block body!), get '/x' do…end route blocks, bare DSL calls — all invisible to extraction at non-body scope except fn-ref candidates + the rails regex extractor |
call (inside method/function bodies) |
visitFunctionBody:5143 | extractCall:3684 → ruby branch 3905-3960 (below) |
alias / alias_method |
no branch / call-at-class-scope | both emit nothing (probed: alias children are name/alias identifier fields, parents not BLOCK_PARENTS) |
comment |
no branch | consumed only by docstring sibling scans |
if/unless/case/while/begin at top level |
no branch | recursed via visitNode — so a require inside a top-level if STILL reaches extractImport (all-visitNode recursion); but calls/identifiers there emit nothing |
uninterpreted (__END__ data) |
no branch | nothing (see grammar prep) |
Not applicable to ruby (verify cheap early-outs): interfaceTypes/structTypes/
enumTypes/typeAliasTypes/propertyTypes/fieldTypes branches; swift property
branch (1121); TS re-export/vue-store export_statement branches (1219/1235);
INSTANTIATION_KINDS (354-361 — no ruby node kinds; ruby .new is handled in
extractCall, so extractInstantiation:4610 is unreachable for ruby);
impl_item (1274); property_signature/method_signature (1282, gated on
TYPE_ANNOTATION_LANGUAGES which excludes ruby, 5752-5754);
extractFilePackage:1397 (no packageTypes → no namespace node);
namespacePrefix always empty (cpp-only).
Node creation, IDs, qualified names
createNode(1308): id =generateNodeId(filePath, kind, name, startRow+1)=`${kind}:${sha256(`${filePath}:${kind}:${name}:${line}`).hex.slice(0,32)}`(tree-sitter-helpers.ts:18-30). FILE node id is literalfile:${filePath}(509). Dedupe/self-checks compare ID STRINGS (node_idsvec pattern).- resolveBody endLine extension (1329) is a no-op (no hook); node endLine =
node.endPosition.row+1 (for a
method, tree-sitter's method node already spans def…end). - contains edge from nodeStack top for every created node (1363); captureValueRefScope on every create (1374).
- qualifiedName = nodeStack names joined
::(buildQualifiedName:1447, namespacePrefix empty) —module Outer; module Inner; class Deep→Outer::Inner::Deep; methodsOuter::Inner::Deep::greet. Compactclass A::B::C/module A::Bkeep the FULL scope_resolution text as the node NAME (extractName → nameField → getNodeText), so nested classes under it get QNs likeA::B::C::m— the::inside the name segment composes verbatim. No receiver-QN path (no getReceiverType). - File node: kind
file, name basename, qualifiedName = filePath, endLine =source.split('\n').length, isExported false.
extractFunction / extractMethod / extractClass for ruby (1517 / 1737 / 1679)
- extractFunction (top-level
method, andsingleton_methodbounced from the 1747 gate): no receiver hook (1522 skipped), name via nameFieldname(identifier; def names with?/!/=suffix —done?,save!,value=— keep the suffix in the name).<anonymous>path (1549) can't trigger (grammar requires a name). No misparse hook. Node extras: docstring (getPrecedingDocstring), signature undefined, visibility (sibling scan!), isExported/isAsync/isStatic/returnType undefined. extractTypeAnnotations → no-op (ruby ∉ TYPE_ANNOTATION_LANGUAGES:5752). extractDecoratorsFor → no-op. Push, walkbodyfield (body_statement) via visitFunctionBody, pop.parameters(method_parameters) are NEVER walked — a call in a default valuedef f(x = compute())emits NOTHING. - extractMethod (defs inside class/module + singleton_method inside
class-like): receiverType undefined (1742); gate 1747 passes via
class-like; the object-literal parent check (1751) never matches ruby
(
object/object_expressionare TS node kinds). extras as function; no receiver QN (1790), no owner-contains fallback (1799 — needs receiverType). extractEnumMembers/interface never. - extractClass: resolvedBody =
bodyfield; skipBodilessClass absent → a bodilessclass TopDoc; end(body field ABSENT — probed) still mints the class node; body-walk target falls back to the CLASS NODE itself (1714) → namedChildren = [name-constant, superclass?] → visitNode on each (emits nothing — but note the superclass subtree is visited AGAIN harmlessly). extras: docstring, visibility (sibling scan), isExported undefined. extractInheritance (1704, below); extractCsharpPrimaryCtorParamRefs/extractDecoratorsFor no-ops; synthesizeMembers absent (1727). Then body children via visitNode with the class pushed. - Nested defs inside a method body: visitFunctionBody:5245 —
methodin functionTypes, named → extractFunction → not class-like at that moment? No: the nodeStack top is the enclosing METHOD node (kind method — not class-like) → extractFunction path… but wait, dispatch inside extractFunction is direct (no gate) → adefnested in a def extracts as a function contained by the enclosing method.class/moduleinside a body:classhits 5255 → extractClass (contained by the method). Amodulenode inside a body is NOT matched in visitForCallsAndStructure (the hook doesn't run there!) — visitFunctionBody never invokes the extractor's visitNode hook, so a module defined inside a method body mints NO module node; its children recurse (5277) and its defs hit 5245 → functions attributed to the enclosing method. Same forincludecalls inside a body: they take extractCall (callTypes) →callsref namedinclude— NOT an implements ref. PRESERVE both.
extractImport (3170) — the every-top-level-call funnel
Hook returns {moduleName, signature} ONLY for require/require_relative-with- string (above). Then:
- import node:
createNode('import', moduleName, node, {signature})— id from kindimport, name = moduleName (json,sidekiq/fetch,../foo/bar). - generic
importsref (3183-3194, hook sets no handledRefs): {fromNodeId: stack top (file/class/module/wherever), referenceName: moduleName VERBATIM, line: call startRow+1, column: call startColumn}. emitRubyRequireRefs(3231-3234 → 3532-3560): re-derives method name + string content itself (namedChildren.find identifier / argument_list / string / string_content — same shapes); req = content text.trim().require_relative→ refPath =path.posix.normalize(dirname(filePath) + '/' + req)(dirname via lastIndexOf('/') on the as-indexed filePath — posix semantics);require→ refPath = req unchanged. Then: no/in refPath → return (bare gem/stdlib require emits NO file ref); append.rbunless already.endsWith('.rb'); push {fromNodeId, referenceName: refPath, referenceKind:'imports', line/col of the CALL}. Sorequire "sidekiq/fetch"emits TWO imports refs (sidekiq/fetch+sidekiq/fetch.rb) after the import node — EMISSION ORDER: node, generic ref, require ref.- Hook null (every non-require call): falls past the python/go/php
multi-import branches to 3350
return— nothing emitted, children still visited by the ladder (skipChildren false for the import branch).
In-body requires NEVER come here (visitFunctionBody routes call to
extractCall) — require "x" inside a def emits only a bare calls ref
require; Kernel.require "some/lib" in a body → calls Kernel.require +
references Kernel (probed). Only visitNode-context requires (top level,
class body, module body, inside top-level if/begin) create import nodes.
extractCall (3684) — the ruby branch (3905-3960), ALWAYS returns early
Reached only from visitFunctionBody:5143. Gate: language==='ruby' && (type 'call' || 'method_call') — before the LITERAL_RECEIVER_TYPES / generic
field_expression machinery, which ruby therefore NEVER runs. Steps:
- methodName =
methodfield text; empty/absent → return with NOTHING (operator/element-reference call shapes; noteelement_referenceis its own node kind and never reaches here anyway). - No
receiverfield → onecallsref {name: methodName, line: call startRow+1, col: call startColumn}. Covers parenless commands,puts,done?-style zero-arg calls,lambda/proc(block bodies then recurse), in-bodyrequire, in-bodyinclude(plaincallsref!). - receiverName = FULL receiver text (
getNodeText— verbatim, sigils and newlines included:@name,@@cv,$g,"literal",5,[1, 2],chained.first_call,a.b(x)). methodName === 'new': className = receiverName after the LAST::(slice(lastIndexOf('::')+2)); if/^[A-Z]/→instantiatesref {name: className} at the call position, return.Widget.new→ instantiatesWidget;NS::Widget.new→Widget(unqualified!);lower.new→ NOT capitalized → falls through to step 5 → calls reflower.new.- SKIP_RECEIVERS = {
self,super} by TEXT (ruby-only set — smaller than the generic {self,this,cls,super,parent,static}): skip → bare methodName; else`${receiverName}.${methodName}`. Safe navigation&.joins with a PLAIN.(deep&.safe_call→deep.safe_call); chains keep raw receiver text (chained.first_call.second_call,Widget.create(x).save→Widget.create(x).save— args text NOT normalized to(), unlike java/php chain encodings); literal receivers are NOT filtered ("literal".upcase→"literal".upcase,5.times→5.times— unresolvable noise, PRESERVE). - Receiver node TYPE exactly
constant(and not skipped) → ADDITIONALreferencesref {name: receiverName, line/col: the RECEIVER's position} —Klass.static_callemits callsKlass.static_call+ referencesKlass; fires for VALUE constants too (RETRY_MAX.times {…}→ callsRETRY_MAX.times+ referencesRETRY_MAX— pinned). Ascope_resolutionreceiver (Foo::Bar.baz) gets NO references ref (type ≠ constant). PRESERVE. - return — inner receiver calls are ALSO visited afterwards (the body
walker recurses children at 5277 — extractCall does not consume its
subtree), so
chained.first_call.second_callemits BOTHchained.first_call.second_callandchained.first_call; interpolation calls inside argument strings emit too (puts "#{@name.upcase}"→ callsputs+ calls@name.upcase). Setter/op-assign LHS calls emit through plain recursion:x.y = 1→ assignment(left: call) → calls refx.y;obj.attr &&= refresh→obj.attr(probed).
Chained-call re-encode (#750, gate list at ~4413), local-variable receiver
inference (#1108) and typed-param receivers (#1125/#1129/#1130) membership:
extraction-side, ruby's involvement is ONLY the recv.method ref shape
above — the ruby branch predates/bypasses the #750 generic re-encode
(ruby is NOT in that gate list; unreachable anyway), and #1108/#1125-#1130
are RESOLUTION-side consumers of lg.log-shaped refs (name-matcher), no
extraction work.
extractVariable (2538) — the python/ruby branch (2709-2727)
kind = 'variable' ALWAYS (no isConst, 2546-2547). docstring computed (2548);
isExported computed → ?? false but NOT passed in the create (create
extra = {docstring, signature} only → isExported undefined on the node —
unlike the TS/Go branches). left = left field ?? namedChild(0); right =
right field ?? namedChild(1). Only identifier or constant LHS mints a
node: name = LHS text, node POSITION = the whole assignment node (id line =
assignment start), signature = `= ${right text .slice(0,100)}` +
'...' when initValue.length >= 100 (UTF-16 slice + length). LHS
left_assignment_list (multiple assignment), instance_variable,
class_variable, global_variable, element_reference, call (setter) →
no node. So: top-level x = 1 AND X = 1 both mint variable nodes;
class/module-scope only CONST = (the 1100 gate); @x/@@x/$x
assignments mint nothing anywhere; method-local assignments mint nothing
(visitFunctionBody has no variableTypes branch). skipChildren → RHS never
walked (no instantiates/calls from initializers); scanFnRefSubtree (1110)
still captures fn-ref containers inside the whole assignment subtree.
Inheritance — extractInheritance for ruby (5291)
Only ONE child type matters: superclass (generic clause branch
5333-5409). The class node's superclass-field child has children
[<(anon), TYPE]; no type_list → targets = [namedChild(0)] → ONE extends
ref {name: the type's FULL text — Bar, A::B::C (scope_resolution
verbatim), even an expression (class Foo < Struct.new(:a) → text
Struct.new(:a) — emit verbatim, resolution drops it); line/col of that
child}. Emitted from extractClass BEFORE the body walk (order: class node →
extends ref → body members). All other clause types in the 5329 loop
(scala/dart/cpp/python-argument_list — gated on node.type
class_definition, ruby's is class → never — go/rust/vbnet/c#/kotlin/
swift/js-heritage/cfml) match nothing. include/extend/prepend →
implements refs come from the visitNode hook (extractor config above), NOT
from here.
Docstrings (tree-sitter-helpers.ts:95-127)
Ruby comments are single comment nodes per # line; =begin…=end is ONE
comment node. Consecutive preceding named siblings accumulate
(unshift → source order), stop at the first non-comment.
DOCSTRING_WRAPPER_TYPES (55-62) contains NO ruby node kinds → the anchor
never climbs. cleanCommentMarkers (77-90): the paired-delimiter branches
don't match (=begin starts with =) → per-line gm strips apply:
^#\s? (the ruby marker), plus the OTHER languages' line strips run too —
an =begin body line starting with * loses it to ^\s*\*\s?, a line
starting -- or // or % is stripped by those rules, and the
=begin/=end lines themselves SURVIVE into the docstring (probed
against the helper logic — pin in the torture fixture). Kernel side:
docstring.rs::preceding_docstring + clean_comment_markers already
implement all of this including the # cleaner — call them, port
nothing; the CRLF ^-after-\r semantics (#1329) are inside
js_multiline_strip (docstring.rs:78). Comment above private def x runs:
the def's siblings are inside the argument_list → no preceding comment → the
CALL swallowed the comment → docstring undefined for the inner def. A
comment between doc and def breaks nothing (comments chain); any other
node type does.
Function-as-value capture (#756) — RUBY_SPEC (function-ref.ts:262-273)
idTypes = EMPTY (bare identifiers are never candidates → explicitRef is
always true, irrelevant since no addressOfOnly). dispatch:
argument_list→args, pair→value(field value). layers:
block_argument→null (fan out namedChildren — &method(:x), &:sym).
special: {call, simple_symbol}. No unwrap/ungatedModes/addressOfOnly.
callspecial (function-ref.ts:700-709): acallVALUE whosemethodfield text is exactlymethod, whoseargumentsfield has EXACTLY 1 namedChild of typesimple_symbol→ candidate name = symbol text minus leading:(method(:cb)→cb). Fires only when the method(...) call sits in a dispatched container:register(method(:cb))(argument_list value) andregister(&method(:cb))(block_argument layer) capture;store = method(:cb)does NOT (assignment is not in ruby's dispatch). Bare-name candidates → the flush gate applies (below).simple_symbolspecial (797-805): symbol in a dispatched container →rubyEnclosingCall(837-843: nearestcallancestor within 4 parent hops) → itsmethodfield text must satisfyisRubyHookCall(284-286):/^(skip_)?(before|after|around)_[a-z_]+$/(282) OR ∈ {validate, set_callback, helper_method, rescue_from} (283). Symbol text (minus:) must match/^[A-Za-z_][A-Za-z0-9_?!]*$/→ candidatethis.<sym>. Coversbefore_action :authenticate_user!(argument_list) andrescue_from E, with: :render_404(pair value; the scope_resolution exception-class arg yields nothing).validates :namedeliberately NOT a hook.skip_before_action :check, only: [:index]→this.checkonly (the array-valued pair normalizes to nothing).- Capture points: visitNode:990, visitFunctionBody:5137, scanFnRefSubtree
(hook-handled subtrees + variable declarations). Container mechanics
(captureFnRefCandidates:408): args mode = every namedChild; pair value =
valuefield. NAME_STOPLIST (121-134) drops self/nil/… candidate names. - Flush gate (flushFnRefCandidates:639): generated-file skip;
this.- prefixed candidates ALWAYS flush (709) — resolution scopes them to the enclosing class + superclasses; baremethod(:x)names need definedHere (same-file function/method names) ∪ importedNames. Ruby imports contribute little:json-style bare requires pass SIMPLE_NAME (661);sidekiq/fetchand…/foo.rbpaths match NEITHER regex (/not in either class, 661/665) → the bare-name gate is effectively "defined in this file" (rust precedent). Survivors dedupe on${fromNodeId}|${name}→ {referenceKind:'function_ref'} refs (FUNCTION_REF_CODE=200 on the wire, buffers.rs:118).
Value-reference edges (398-931) — ruby IS in VALUE_REF_LANGS (401)
Port the full machinery (crib python.rs — python is also a member):
CODEGRAPH_VALUE_REFS=0 kill; MAX_VALUE_REF_NODES=20_000 caps prune scan and
each reader scan; isGeneratedFile skip.
- Targets (captureValueRefScope:735): created nodes of kind
constant/variable — for ruby ALWAYS 'variable' — name length ≥3 AND
/[A-Z_]/(snake_case with_qualifies:top_varis a target), parent scope id prefixfile:|class:|module:|struct:|enum:— ruby hits file/class/module (the comment at 747 names Ruby as the class/module-scope motivation). Last-write-wins map + per-name counts. - Reader scopes: every function/method/constant/variable node (764).
- Shadow prune (803-878): DFS with the per-grammar declarator switch — the
ruby-relevant case is
assignment(829-834): left =leftfield ??pattern?? namedChild(0); left typeidentifier→ bump it; else bump every namedChild of left (multiple-assignx, y = …bumps both;constant-typed LHS has NO named children → constants are NEVER counted → never pruned — only identifier-named targets can be shadowed away). bump() counts only identifier/simple_identifier nodes whose text is a target (805-811).declCount > fileScopeCount→ target deleted. - Emission (880-930): per reader scope DFS; reader node types
identifierandconstant(897-908 — theconstantentry exists FOR ruby: both a constant's def and its reads areconstantnodes);name/simple_identifierare php/kotlin-only, inert. Skip self-id, same-name, dedupe per (scope,target) → EDGE {kind:'references', metadata:{valueRef:true}} — edges, not unresolved refs, appended AFTER the walk (flush order below). Both DFSs are STACK-based (push namedChildren in order, POP from the end) → statements are visited in REVERSE source order, and the value-ref EDGE order follows — pinned: a method readingTOP_LIMITon line 13 andRETRY_MAXon line 14 emits the RETRY_MAX edge FIRST (extract-vref.txt). python.rs already reproduces this traversal — crib it, don't "fix" to preorder.
Misc shared paths
- Positions:
line = startPosition.row + 1,column = startPosition.column— UTF-16 code units (textutil::col16), as arestartIndex/endIndexsubstrings,.trim()s and.slice(0,100)truncations (signature). - Refs carry NO filePath/language (store denormalizes) — wire contract is
exactly extractFromSource's return.
implementsis a normal EdgeKind ref code;function_ref= code 200. extract()wraps: file node first, nodeStack=[fileId], no packageNode; flushFnRefCandidates then flushValueRefs at the very end (538-539) — so table order is: file node → walk-order nodes; contains edges interleaved with creation → value-ref edges LAST; walk-order refs → function_ref refs appended at flush. The harness/store are rowid-order-sensitive — reproduce this exact order.- extractStaticMemberRef (4750) early-outs on STATIC_MEMBER_LANGS (4751, ruby out); extractVariableTypeAnnotation (6074) + extractTypeAnnotations gated off (ruby ∉ TYPE_ANNOTATION_LANGUAGES:5752); no cpp fn-ptr/stack paths; no INSTANTIATION_KINDS.
- Parse errors:
has_error()→defer:(expect ~0% incidence, §Grammar prep). tree.delete()/source-release are wasm-side concerns. - CRLF hazards inventory for the ruby path: the ONLY multiline regex ruby
exercises is cleanCommentMarkers' strip set — already CRLF-correct via
js_multiline_strip(docstring.rs, #1329). ruby.ts itself has no regexes over source (charCode checks + string equality only); emitRubyRequireRefs uses.trim()(JS trim == Rust trim for \r/\n/space/tab — but JS trims U+FEFF/NBSP too; a BOM inside a require string is unreachable in practice, note only). The rails framework regexes stay TS-side. Grammar-level CRLF probed clean (heredocs, %-literals, comments — no error, same shapes).
Frameworks that consume ruby extraction artifacts (stay TS-side)
railsResolver (resolution/frameworks/ruby.ts) — detect: Gemfile 'rails' /
config/application.rb / app/controllers/application_controller.rb /
config/routes.rb.
extract()(109-190, regex over raw source AFTER either arm — no port, but the merge shape must hold): for.rbfiles only, emitsroutenodes with id`route:${filePath}:${line}:${METHOD}:${path}`(NOT hashed) for explicitget/post/... '/p', to: 'c#a'routes and RESTfulresources/resourceexpansions, plus onereferencesrefcontroller#actionper route FROM the route node (framework refs DO carry filePath+language, unlike extraction refs). Merged at tree-sitter.ts:6746-6747 (nodes + references appended after extraction).- Extraction-side emissions the port MUST reproduce for ruby resolution to
keep working: (a) the require-path
importsrefs (…/….rbshapes — resolved by file-path suffix matching); (b) the mixinimplementsrefs (module composition edges); (c)recv.methodcalls refs (local-variable receiver inference #1108 + rails Model resolution consume these); (d) the constantreferencesrefs from capitalized receivers (Pattern 1 model resolution feeds on constant refs); (e)this.<sym>function_ref refs (hook-DSL → class-scoped resolver).claimsReference(18-20) claimscontroller#actionshapes — produced by the framework extractor itself, not the walker. - No sinatra-specific resolver exists; sinatra route blocks (
get '/x' do) are top-level calls → invisible to extraction (importTypes funnel) and NOT covered by railsResolver's regexes (which need theto:/=>controller form). Correct today's behavior — don't "fix" in the walker.
Gates (per plan §5, no exceptions)
- Standalone GRAMMAR-BUMP gate first (rust pattern), before any walker:
vendor wasm +
VENDORED_WASM_LANGS += 'ruby'(+ Cargo pin staged with it), then old-vs-new full-init dump-diffs (scripts/dump-graph.mjs, cmp) on the three gate repos with the kernel OFF both arms (pure wasm-path bump isolation). Expected: byte-identical on sinatra/jekyll EXCEPT the three sinatra__END__files (inert — but__END__shifts no emissions, so expect byte-identical there too); rails differs ONLY in where_clause.rb-class safe-nav-operator refs (recv.!→recv.!=, precision-positive — classify every hunk). Full suite green ×2. - Torture fixture
torture.rb(+ CRLF variant derived in-memory, kernel-tsjs-parity pattern), pinning at minimum: module nesting + class in module (QN join) + compactclass A::B::C+module A::B; superclass plain + scoped (< A::B::Cfull-text extends) + expression superclass (< Struct.new(:a)verbatim); bodilessclass X; end; include/extend/prepend single + multi-arg +extend self+ receiver-formFoo.include Bar(nothing);class << self(methods → outer class) +def self.xin class (method) and at top level (function);privatebare (invisible) +private :sym(poisons all later defs) +private def(inner def public, later defs private) +public def; def with?/!/=-suffixed names; CONST assignment at file/class/module scope (kind variable, signature= …with the 100-slice) +x = 1at file scope (node) vs class scope (nothing) +@x/@@x/$x(nothing) + multiple assignment (nothing) + operator_assignment (nothing) +X = Foo.new(NO instantiates); requires: bare (json— import node + 1 ref), slashed, require_relative with.., in-body require (calls ref only),Kernel.requireat top level (import node — the find-identifier quirk),require %q(pct/lib)(full require),require "interp/#{x}"(first-segment refsinterp/+interp/.rb),require :sym(nothing); calls: parenless command, bare identifier statement (do_block vs brace block block_body!), modifier-guarded bare identifier (nothing),done?in condition, self./super receivers,@ivar.m/@@cv.m/$g.m, literal receivers ("s".upcase,5.times),Klass.m(+references),NS::Klass.m(no references),Widget.new/NS::Widget.new(instantiates Widget)/lower.new(callslower.new), chains parenless + with-args (a.b(x).cverbatim), safe-nav&.(+ the&.!=operator-method form — the one grammar-delta shape), setterx.y = 1(callsx.y),h[:k] = v(nothing), interpolated"#{a.b}"call + interpolated bare identifier (nothing), heredoc<<~with interpolated call,yield,lambda {}/proc {}/->() {}; begin/rescue(exceptions + bare)/else/ensure/case-when(+then)/while bare-identifier statements (refs) vs block_body (none); fn-ref shapes:before_action :sym+skip_before_action :sym, only: [:syms]+around_create+rescue_from E, with: :sym(pair) +validates :attr(nothing) +helper_method/validate/set_callback;register(method(:cb))+register(&method(:cb))+store = method(:cb)(nothing) +each(&:sym)(nothing) + gate cases (cb defined in-file vs not); define_method at class scope (fully invisible incl. body) ;alias+alias_method(nothing); attr_accessor/reader/writer (nothing); nested def in def (function), class-in-method-body (extracted), module-in-method-body (NO module node); docstrings:#runs,=begin/=end(markers survive), comment aboveprivate def(lost), comment-above-class;__END__+ data; value-refs: file CONST read from a method (constant-node reader), class-scope CONST + module-scope CONST, snake_caselower_conftarget + a locallower_conf = …shadow inside a def (prune), conditional double-def (kept); a def AFTERattr_accessor-style calls (visibility scan walks through them). - Parity sweeps (
scripts/kernel-parity.mjs, order-sensitive full-object): sinatra (small, 147 rb files), jekyll (medium, 164), rails (large, 3,452) — gate-repo clones from the survey lived at the prior session's scratchpad (…/765a9532…/scratchpad/{sinatra,jekyll,rails}; re-clone fresh if gone — agent-eval policy, public OSS only). Then full-init dump-diffs byte-identical (kernel arm vsCODEGRAPH_KERNEL=0,dump-graph.mjs, cmp) on the same three. - Deferral-rate guard: default
--max-deferral 0.1and expect ~0 — measured 0.00% parse-error incidence on all three repos, both grammars. Any deferral on a ruby sweep is a walker bug signal, not grammar reality. - Suite: new
__tests__/kernel-ruby-parity.test.ts(torture + CRLF-derived variant + one intentionally-erroring defer fixture — e.g. an uncloseddef— asserting the wasm fallback path); full suite green ×2 withCODEGRAPH_KERNEL_EXPECT=1. DEFAULT_ROUTED += ruby(kernel/index.ts:37) only after ALL of the above; changelog rides the existing kernel entry.- Post-route perf sanity: remember §arch-2 — a real Rails app forces the decoded path (framework extract()), the gate repos measure the raw transport; measure each on the matching repo class.
Fixtures to build
__tests__/fixtures/kernel-parity/torture.rb— the full inventory above (every visitNode-dispatch row, every extractCall shape, every fn-ref/ value-ref/visibility/docstring case,__END__trailer last).- CRLF variant — derived in-memory in the parity test (normalization-proof,
the kernel-tsjs-parity pattern), asserting byte-parity kernel↔wasm on CRLF
bytes (heredoc +
#-run +=begindocstring cleaning under CRLF). - Defer fixture — a
.rbwith a genuine parse error (uncloseddef/endmismatch), pinning: kernel defers (defer:), wasm output is served, and the file is absent from deferral-rate failures at the fixture scale. - A minimal Rails-app-shaped fixture for the decoded-path seam (a
config/routes.rb+ one controller underapp/controllers/) exercising framework extract() merge over a kernel-extracted file — route nodes +controller#actionrefs land on top of walker output identically in both arms (can live in the existing frameworks-integration suite if simpler).