github-featured/deusdata--codebase-memory-mcp
2
0
Fork 0
Code Issues Pull Requests Deployments Models Agent Notes Packages Projects Releases Wiki Activity
2,472 Commits 300 Branches 52 Tags
main
Commit Graph

3 Commits

Author SHA1 Message Date
Martin Vogel 3ee9d5d112 Drop the nim grammar and refresh language counts 
Removes the heaviest vendored grammar (66 MB) along with its language
wiring and test fixtures; the supported-language count moves to 158
across README, site, and manifest. The grammar manifest additionally
records the canonical-source decisions for the five
registry-disagreement grammars and the per-directory license files
restored earlier this week.
 2026-06-12 02:17:39 +02:00
vinay-veerappa 048f1b46c9 feat(lang): add Pine Script support via vendored kvarenzn/tree-sitter-pine (#273) 
Adds Pine Script (TradingView's indicator/strategy DSL) as a first-class
language using the project's existing language-add pipeline:

- scripts/new-languages.json: register pine (display "PineScript",
  ts_func tree_sitter_pine, repo kvarenzn/tree-sitter-pine, extension
  .pine, has_scanner=true, module_root source_file)
- internal/cbm/vendored/grammars/pine/: vendored from
  https://github.com/kvarenzn/tree-sitter-pine via scripts/vendor-grammar.sh
  — parser.c, scanner.c, tree_sitter/ headers. License is ISC (declared
  in upstream package.json; written into vendored LICENSE for clarity
  since upstream has no LICENSE file)
- internal/cbm/grammar_pine.c: 4-line wrapper generated by
  scripts/generate-lang-code.py wrappers
- internal/cbm/cbm.h: CBM_LANG_PINE before CBM_LANG_COUNT
- internal/cbm/lang_specs.c: 7 node-type arrays (func / class /
  module / call / var / branch / assign) cross-checked against
  kvarenzn's grammar.js, plus the spec table entry
- src/discover/language.c: .pine -> CBM_LANG_PINE extension dispatch
  and "PineScript" name

scripts/audit-grammar-security.sh on the vendored grammar: no
dangerous includes (only string/stdint/stdbool/stdlib), no system /
popen / exec / socket / mmap / dlopen calls, no constructor/destructor
attributes, no inline asm. The 148-line scanner.c only handles
INDENT/DEDENT/NEWLINE tokens via the standard tree-sitter lexer API.

Distilled from #273 rather than vendored verbatim because the
original PR's parser.c was a plain 46k-line drop without provenance,
no LICENSE attribution, and bundled an unrelated Makefile change
(scripts/embed-frontend.sh -> sh scripts/embed-frontend.sh) that
didn't belong to a Pine Script feature. The node-type choices in
the lang spec match the original PR's selections, all of which were
verified to be real grammar.js rules.

Full suite: 2840 passed, 0 failed under ASan + UBSan.

Closes #273.

Co-authored-by: vinay-veerappa <vinay.veerappa@gmail.com>
 2026-05-09 11:50:05 +02:00
test b2b48f8b0d Add 89 new tree-sitter grammars (66 to 155 languages) 
Vendor, wire up, and compile 89 new tree-sitter grammars, expanding
language support from 66 to 155 languages. All grammars pass security
audit (no dangerous patterns in scanner.c files).

New programming languages (31):
  Solidity, Typst, GDScript, Gleam, PowerShell, Pascal, D, Nim, Scheme,
  Fennel, Fish, AWK, Zsh, Tcl, Ada, Agda, Racket, Odin, ReScript,
  PureScript, Nickel, Crystal, Teal, Hare, Pony, Luau, Janet, Sway,
  NASM, Assembly, TLA+, Pkl, Cairo, Move, Squirrel, ISPC, FunC, Smali

New config/data/IDL formats (31):
  Just, Astro, Blade, Go Template, Templ, Liquid, Jinja2, Prisma,
  Hyprlang, DotEnv, Diff, WGSL, KDL, JSON5, Jsonnet, RON, Thrift,
  Cap'n Proto, Properties, SSH Config, BibTeX, Starlark, Bicep, CSV,
  Requirements, HLSL, VHDL, SystemVerilog, DeviceTree, Linker Script,
  GN, Kconfig, BitBake, TableGen, Slang, LLVM IR, Smithy, WIT,
  Go Mod, Mermaid, RST, Beancount, Puppet, PO, Regex, JSDoc,
  gitattributes, gitignore, Apex, SOQL, SOSL

Infrastructure:
  - scripts/new-languages.json: manifest for all new languages
  - scripts/generate-lang-code.py: generates boilerplate from manifest
  - scripts/audit-grammar-security.sh: pre-vendoring security scanner
  - Fixed angle-bracket includes in 18 grammars
  - Fixed PureScript scanner const mismatch
  - Fixed VHDL scanner void* API signatures
  - Fixed RST tree_sitter_rst/ subdirectory include paths
  - Copied crystal unicode.c (extra scanner dependency)

All new languages start with minimal lang specs (module_types only).
Function/class/call extraction specs to be refined incrementally.
 2026-04-15 21:28:51 +02:00