Commit Graph

385 Commits

Author SHA1 Message Date
Nir Paz 1210b82237 fix: harden concealed instruction detection
Signed-off-by: Nir Paz <npaz@nvidia.com>
2026-08-24 14:00:23 -07:00
github-actions[bot] cc393efe9d Merge branch 'main' into codex/security-text-normalization 2026-08-24 18:41:21 +00:00
benkwok d486d0a84d fix(pe3): env lookbehind and singular reference dir (#415)
* fix(pe3): env lookbehind and singular reference dir

Two independent PE3 false-positive fixes, both confirmed against the
official anthropics/skills repo (mcp-builder):

- The .env pattern had no lookbehind, so it matched Python attribute
  access (self.env, args.env) as if it were a dotenv file reference.
  Added (?<!\w) so it only fires when .env is not part of an identifier.

- _PE3_TOKEN_DOCUMENTATION_DIRS only recognized the plural "references",
  so the existing OAuth access-token exemption silently failed to apply
  under a singular reference/ directory (mcp-builder/reference/). Added
  singular forms for docs/procedures/references/examples/guides.

Both fixes verified against the existing test suite for
_is_env_file_reference_in_docs and _is_pe3_documentation_example, and new
regression tests added for the attribute-access and singular-directory
cases, including negative-space tests confirming real credential-theft
instructions with the same vocabulary still fire.

Closes #406, #407

Signed-off-by: Benedict Kwok <bkwok.oracle@gmail.com>

* test(pe3): match #393's contextual-triage tagging contract

main's Security fixes (#393) changed PE2-PE5's doc-context suppression
from hard-dropping the finding to tagging it contextual-triage /
likely-benign-context and keeping it. Update the two new reference-dir
tests added in this branch to assert on the tag instead of absence,
matching the pattern #393 already applied to the equivalent
negated-credential-access test.

Signed-off-by: Benedict Kwok <bkwok.oracle@gmail.com>

---------

Signed-off-by: Benedict Kwok <bkwok.oracle@gmail.com>
Co-authored-by: Benedict Kwok <bkwok.oracle@gmail.com>
Co-authored-by: Narendran Raghavan <32655573+rng1995@users.noreply.github.com>
2026-08-24 18:41:07 +00:00
github-actions[bot] 42f0b4f68b Merge branch 'main' into codex/security-text-normalization 2026-08-24 18:29:29 +00:00
Deepak Jain (DJ) 083522a550 feat(llm): support configurable output language (#425)
* feat(llm): support configurable output language

Signed-off-by: Deepak Jain <deepujain@gmail.com>

* fix(llm): validate output language label

Signed-off-by: Deepak Jain <deepujain@gmail.com>

---------

Signed-off-by: Deepak Jain <deepujain@gmail.com>
2026-08-24 18:29:15 +00:00
github-actions[bot] 41bcdaf3f2 Merge branch 'main' into codex/security-text-normalization 2026-08-24 18:24:30 +00:00
UmranPros 837d100d6f feat: implement dynamic analyzer discovery and risk score validation (#74)
* feat: implement dynamic analyzer discovery and risk score validation

Resolves TODO A.2.1-A.2.4 (Analyzer discovery) and TODO A.3.2 (Risk score assertion).

Signed-off-by: umran666 <shaikumran666@gmail.com>

* fix: address PR review feedback on dynamic discovery

Signed-off-by: umran666 <shaikumran666@gmail.com>

---------

Signed-off-by: umran666 <shaikumran666@gmail.com>
Co-authored-by: Narendran Raghavan <32655573+rng1995@users.noreply.github.com>
2026-08-24 18:24:08 +00:00
Narendran Raghavan 2d346c0b07 Merge branch 'main' into codex/security-text-normalization 2026-08-24 11:15:29 -07:00
Burak Bayır a13e28ec26 fix(scan): exclude inactive Git hook samples (#412)
Signed-off-by: kriptoburak <kriptoburak@users.noreply.github.com>
Co-authored-by: kriptoburak <kriptoburak@users.noreply.github.com>
Co-authored-by: Narendran Raghavan <32655573+rng1995@users.noreply.github.com>
2026-08-24 18:15:18 +00:00
github-actions[bot] b10b82eaa3 Merge branch 'main' into codex/security-text-normalization 2026-08-24 18:07:24 +00:00
benkwok 8e20d0ecf1 fix(ea1): bound wildcard-tool-access match to a single line and a standalone asterisk (#417)
Two independent EA1 false positives, both confirmed against the official
anthropics/skills repo (mcp-builder/SKILL.md):

- The pattern used bare \s* between the colon and the expected wildcard
  value. Python's \s matches newlines, so the gap could span a blank line
  and bridge two unrelated headings ("For each tool:" + blank line +
  "**Input Schema:**"). Changed to [ \t]* so the match is bounded to a
  single line.

- Nothing required the matched * to be a standalone token, so the first
  * of a closing ** bold-markdown span satisfied it ("**API Coverage vs.
  Workflow Tools:**" matched as "Tools:*"). Added a negative lookahead
  (?!\*|\w) so the asterisk must not be immediately followed by another
  asterisk or a word character.

New test file added (test_ea1_wildcard_line_boundary.py) covering both
false-positive cases plus regression tests confirming genuine single-line
wildcard grants (tools: "*", tools: [*], permissions: '*', tools: *)
still fire.

Testing:
- make test (unit + integration): 2804 passed, 31 passed, 0 failed
- make lint: clean
- make format: clean

Closes #405

Signed-off-by: Benedict Kwok <bkwok.oracle@gmail.com>
Co-authored-by: Benedict Kwok <bkwok.oracle@gmail.com>
2026-08-24 11:07:10 -07:00
Nir Paz 1d50da6a70 fix(security): fail closed on obfuscated instructions
Signed-off-by: Nir Paz <npaz@nvidia.com>
2026-08-21 17:53:40 -07:00
Nir Paz e6a96df04e fix(security): bound stable Unicode projection
Signed-off-by: Nir Paz <npaz@nvidia.com>
2026-08-21 15:50:44 -07:00
Nir Paz d8c3213871 test(security): cover ambiguous spaced instructions
Signed-off-by: Nir Paz <npaz@nvidia.com>
2026-08-21 15:34:38 -07:00
Nir Paz 057fd7b2f7 fix(security): detect inter-character text obfuscation
Signed-off-by: Nir Paz <npaz@nvidia.com>
2026-08-21 15:22:43 -07:00
Nir Paz 10923e0d55 fix(security): normalize concealed instruction text
Signed-off-by: Nir Paz <npaz@nvidia.com>
2026-08-21 12:47:52 -07:00
Amir Fathi 698e2bf29c fix(report): flag a partial LLM failure as degraded, not only a total one (#362)
* fix(report): flag a partial LLM failure as degraded, not only a total one

_llm_runtime_status() only set degraded when every LLM call failed
(succeeded == 0). A rate-limited provider that drops a single batch
(e.g. semantic_security_discovery hits a 429) still has succeeded > 0,
so the scan reported a normal risk_assessment even though the
security-critical analyzer never ran. Widen the condition to
succeeded < attempted, so any dropped batch degrades the scan and
the existing fail-closed floor (CAUTION instead of SAFE) applies to a
partial pass too. Updated the two degraded-scan messages to say how
many of the calls failed instead of assuming all of them did.

Covers request 3 of #303 (surface incompleteness in the verdict).
Request 1 (configurable concurrency) shipped in #305; request 2
(retry with backoff) is left to the already-open #29.

Refs #303

Signed-off-by: Amir Fathi <amirfathi.me@gmail.com>

* fix(report): mark a batch record failed on any dropped batch, and stop meta-analysis fields inheriting other analyzers' failures

Two gaps from review on #362:

1. llm_call_log records were built with
   ok=bool(outcome.successful) or not outcome.failures, so an analyzer with
   one succeeded batch and one dropped/429'd batch still recorded ok=True.
   In that exact case succeeded == attempted at the report layer and the
   scan stayed SAFE, defeating the partial-coverage fix. Now the record is
   ok=not outcome.failures: any dropped batch marks the whole record failed.
   Applied identically in the three semantic analyzers and meta_analyzer,
   the four call sites that build this record.

2. meta_analysis_applied and the llm_available field were derived from the
   aggregate `degraded` flag, which pools every LLM-backed node together.
   That let a different analyzer's dropped batch force
   meta_analysis_applied=False, filtering_mode="heuristic" and
   llm_available=False even when meta_analyzer's own call fully succeeded,
   misstating two independent contracts (meta-analysis ran vs. some
   coverage was lost) as one boolean. Both fields now derive from
   is_llm_available() plus meta_analyzer's own llm_call_log record only;
   the coverage loss from other analyzers still surfaces through
   llm_degraded / llm_calls_attempted / llm_calls_succeeded, unchanged.

Verified: test_partial_batch_failure_records_llm_failure (renamed from
..._records_llm_success, now pins ok=False) and three new report-level
tests, run red against the pre-fix code (3 of 4 failed) and green after.
tests/nodes/test_report.py: 66 passed. Full suite in Docker
(python:3.12-slim): 1947 passed, 13 skipped, 4 xfailed, 0 failed. ruff
lint and format-check both pass.

Signed-off-by: Amir Fathi <amirfathi.me@gmail.com>

* fix(report): require an actual meta_analyzer record for meta_analysis_applied

all([]) is True on an empty list, so an empty meta_analyzer_records left
meta_analysis_applied True even when meta_analyzer made no call at all (the
no-findings path, where it short-circuits to not_applicable). That still
violated the "did meta-analysis actually run" contract from the prior
review.

meta_analysis_applied now requires at least one meta_analyzer record and
all of them ok. llm_available is unchanged: provider availability is a
separate contract from whether meta_analyzer had anything to do, and it
stays vacuously true when meta_analyzer never ran.

Adds a regression covering the no-findings/no-record case, asserting
meta_analysis_applied is False while llm_available stays True.

Signed-off-by: Amir Fathi <amirfathi.me@gmail.com>

---------

Signed-off-by: Amir Fathi <amirfathi.me@gmail.com>
2026-08-21 12:01:19 -07:00
Narendran Raghavan 73dd1f182a Security fixes (#393)
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
2026-08-21 10:08:55 -07:00
Rod Boev c94b50a6f5 fix(scan): port transitive scanning to current main contracts (#225)
Signed-off-by: Rod Boev <rod.boev@gmail.com>
Co-authored-by: Narendran Raghavan <32655573+rng1995@users.noreply.github.com>
2026-08-21 00:11:09 +00:00
zhu a79028b18b fix: fail closed when LLM provider is unavailable (#291)
Signed-off-by: tuodijihua <158809980+tuodijihua@users.noreply.github.com>
Co-authored-by: Narendran Raghavan <32655573+rng1995@users.noreply.github.com>
2026-08-20 16:58:13 -07:00
Marco Macrì 0321f8c7d9 fix(cli,supply-chain): parse package.json as JSON, and send fatal errors to stderr (#323)
* fix(supply-chain): parse package.json as JSON

package.json was scanned line by line. A manifest written on a single line —
valid JSON, and what several generators emit — never entered the dependency
section, so it produced *no* dependencies at all and the file passed silently.
That is not noise, it is blindness: the scanner reports nothing and the caller
cannot tell the difference from a clean manifest.

It is now parsed as JSON. Version extraction is unchanged, including the caret
handling: only the parsing changes. Line numbers survive the switch — the entry
is located from the section header onwards, so a name that also appears in
"scripts" does not steal the position — and a manifest that does not parse
still falls back to the previous scan rather than going blind.

Tests: one-line manifest, compact manifest, line numbers preserved, a name
shadowed by "scripts", invalid JSON falling back, a non-object manifest, and
non-string specs ignored.

Signed-off-by: Mark2Mac <Mark2Mac@users.noreply.github.com>

* fix(cli): route fatal diagnostics to stderr

Anything driving the CLI from a script separates the two streams and parses
stdout. A diagnostic printed there is lost as a diagnostic — a failed scan left
an empty error log and nothing to act on — and corrupting as output, since it
lands in the same stream as the report.

The mechanism already exists: err_console arrived with the author-shipped
baseline notices, which correctly go to stderr. This commit only moves the
diagnostics onto it. Thirteen call sites: every message that prints and then
raises typer.Exit, the two print_exception() calls in the --verbose branches,
and the per-skill error inside the multi-skill loop. --version stays on stdout,
because that is program output rather than a diagnostic.

Tests enumerate all thirteen paths and assert the message reaches stderr and
never stdout. Verified red against the unmodified module: fourteen failures.

The last test is the reason the others are not enough. Twelve of these sites
already existed when this change was first written and it moved only eight of
them; a thirteenth arrived later, in the same commit that introduced
err_console. The invariant has no enforcement, so it regenerates. The test
parses cli.py and fails when error-styled output is written to the default
console.

Signed-off-by: Mark2Mac <Mark2Mac@users.noreply.github.com>

---------

Signed-off-by: Mark2Mac <Mark2Mac@users.noreply.github.com>
Co-authored-by: Mark2Mac <Mark2Mac@users.noreply.github.com>
Co-authored-by: Narendran Raghavan <32655573+rng1995@users.noreply.github.com>
2026-08-20 16:54:04 -07:00
Rod Boev b44c94c4af fix(analyzer): filter license boilerplate from EA3 static findings (#312) (#328)
* fix(analyzer): filter license boilerplate from EA3 static findings (#312)

Signed-off-by: Rod Boev <rod.boev@gmail.com>
(cherry picked from commit eae96fe3c697dad995313aebf4cf0cf2ea303c1e)

* fix(analyzer): scope EA3 license suppression to recognized boilerplate (#312)

Only suppress an EA3 finding on a text-like legal basename when the matched
line is recognized license boilerplate content, so instructions smuggled into
license-named files stay reported.

Signed-off-by: Rod Boev <rodboev@users.noreply.github.com>
(cherry picked from commit 25d7f9d25c2f2deb7dbbf61629a1503b4dd24faf)

* fix(analyzer): require canonical license ranges for EA3 suppression

Signed-off-by: Rod Boev <rod.boev@gmail.com>
(cherry picked from commit c77cadcab0829062db3d1faa03f334007cdd3a7a)

* test(analyzer): validate canonical license fixtures independently

Signed-off-by: Rod Boev <rod.boev@gmail.com>
(cherry picked from commit 34d794e394bacd2cfdb62c48729e4d0b083ac7c2)

* fix(analyzer): bound license ranges around EA3 matches

Signed-off-by: Rod Boev <rod.boev@gmail.com>
(cherry picked from commit abde68ca1341f40dfb3597c91a12806057c1e65d)

* test(analyzer): isolate legal filename coverage

Signed-off-by: Rod Boev <rod.boev@gmail.com>
(cherry picked from commit e751f7b632ba5d5def2011edf3cc6884e81bdd40)

---------

Signed-off-by: Rod Boev <rod.boev@gmail.com>
Signed-off-by: Rod Boev <rodboev@users.noreply.github.com>
Co-authored-by: Narendran Raghavan <32655573+rng1995@users.noreply.github.com>
2026-08-20 16:45:54 -07:00
badhope 1d379dca7f fix(analyzer): make E2 whitespace-tolerant and detect all os.environ read forms (#331)
* fix(analyzer): make E2 whitespace-tolerant and detect all os.environ read forms

The E2 regex fallback (used when Python source cannot be parsed by AST)
was missing several common os.environ access patterns and was not
whitespace-tolerant for the patterns it did cover.

Add fallback patterns for:
- os.environ['KEY'] / os.environ["KEY"] (whitespace-tolerant)
- os.environ.get('KEY') (whitespace-tolerant)

All existing patterns (items(), copy(), dict(), {**} spread) retain
whitespace tolerance. The dict-spread regex explicitly requires braces
({**os.environ}) so bare exponentiation (2 ** os.environ) is not
flagged as environment harvesting.

AST-level detection (used when Python parses successfully) now also
covers:
- os.environ['KEY'] / os.environ["KEY"] via ast.Subscript handling
- os.environ.get('KEY') by adding 'get' and 'setdefault' to the
  _ENVIRONMENT_MAPPING_METHOD_CONFIDENCE mapping

This closes the gap where whitespace-obfuscated access
(e.g. `os . environ [ 'API_KEY' ]`) was parsed by the AST but not
emitted as a finding because Subscript nodes were not checked and
the 'get' method was not in the confidence table.

Add regression tests:
- Whitespace-obfuscated environ access is detected (>= 2 findings)
- 2 ** os.environ (exponentiation) is NOT flagged as E2
- {**os.environ} (dict spread) IS flagged as E2

Signed-off-by: badhope <weed33834@users.noreply.github.com>

* fix(analyzer): scope E2 to full-environ reads, drop single-key lookup flagging

Upstream treats a targeted single-key lookup (os.environ['KEY'],
os.environ.get('KEY')) as distinct from environment harvesting, and the
rebased test_patterns.py asserts those forms must NOT emit E2. Drop the
single-key detection added earlier (regex patterns, method-confidence
entries for get/setdefault, and the Subscript AST branch) while keeping
whitespace-tolerant detection of full-environ reads (copy/items/keys/
values, dict(os.environ), {**os.environ}) and the brace-bounded dict-spread
pattern that avoids flagging 2 ** os.environ.

Signed-off-by: weed33834 <weed33834@users.noreply.github.com>

---------

Signed-off-by: badhope <weed33834@users.noreply.github.com>
Signed-off-by: weed33834 <weed33834@users.noreply.github.com>
Co-authored-by: badhope <weed33834@users.noreply.github.com>
Co-authored-by: Narendran Raghavan <32655573+rng1995@users.noreply.github.com>
2026-08-20 16:41:57 -07:00
Narendran Raghavan 51ed188bef Inspect concealed executables in nested artifacts (#382)
* Inspect concealed executables in nested artifacts

Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>

* fix: harden nested artifact inspection boundaries

Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>

---------

Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-08-20 23:38:25 +00:00
Kiell Tampubolon c85815836b fix(scanner): prevent SKILL.md findings from being hard-dropped by is_code_example (#373) (#381)
is_code_example() had no SKILL.md exclusion, unlike _is_documentation_context()
which already special-cases it. Since SKILL.md's file_type ("markdown") is
non-executable, any finding within 3 lines of an indicator like "for example"
or a backtick fence was silently dropped in static_runner._scan_path -
including HIGH-confidence prompt-injection findings on the primary attack
surface.

Add the same SKILL.md guard to is_code_example() via an optional path kwarg,
and pass the file path at the one call site that hard-drops on it. A
pre-existing anti_refusal test asserted the vulnerable behavior for AR1 on
SKILL.md; it is split into a generic-markdown case (still downgraded) and a
SKILL.md case (now correctly preserved).

Signed-off-by: glatinone <93207632+glatinone@users.noreply.github.com>
Co-authored-by: Narendran Raghavan <32655573+rng1995@users.noreply.github.com>
2026-08-20 16:34:51 -07:00
Andrew White 9d57e26d45 fix: assert combined report file exists in recursive scan test (#384)
Signed-off-by: andrewwhitecdw <andrewwhitecdw@users.noreply.github.com>
Co-authored-by: andrewwhitecdw <andrewwhitecdw@users.noreply.github.com>
Co-authored-by: Narendran Raghavan <32655573+rng1995@users.noreply.github.com>
2026-08-20 23:31:06 +00:00
Marco Macrì 14ec304bcd fix(nv_build): retire the dead models and point the default at a served one (#391)
* fix(nv_build): declare glm-5.2's real limits so calls stop failing

The bundled registry gave z-ai/glm-5.2 a 1000000-token context window and no
output cap. model_info derives the output budget as
ctx * (1 - MAX_INPUT_TOKENS_PCT), so every request asked for 250000 output
tokens and the endpoint answered:

  400 This model configuration accepts at most 202749 combined input and output
      tokens. However, your request has 1249 input tokens and asks for 250000
      output tokens (251249 tokens total).

202749 is quoted verbatim by the endpoint in that 400. With the entry corrected
the same scan completes with 4/4 LLM calls and the meta-analyzer applied.

An over-stated context window does not degrade gracefully: it zeroes the LLM
stage, and nothing in the error points at the registry. Under-stating is safe,
over-stating is not.

Limits may vary per account, which is now noted in the YAML.

Scope is deliberately one entry. The registry also names three models the
catalogue no longer serves, but removing them is coupled to DEFAULT_MODEL by an
invariant the suite already asserts ("nv_build's default model is in its
registry"), so that change travels with the default in a separate PR.

Refs #388

Signed-off-by: Mark2Mac <Mark2Mac@users.noreply.github.com>

* fix(nv_build): retire the dead models and point the default at a served one

Depends on #390.

Three registry entries name models GET /v1/models does not serve:
deepseek-v4-flash (410 Gone since 2026-08-07), deepseek-v4-pro, and glm-5.1.
One of them is DEFAULT_MODEL, and another is the meta_analyzer slot override,
so with no SKILLSPECTOR_MODEL set the out-of-the-box path failed every call.

Removing them and retargeting the default is ONE change, not two. The suite
already asserts the invariant that couples them, in test_constants:

  "nv_build's default model is in its registry — no warnings expected"

Dropping the default from the registry while leaving it as the default breaks
that test, and it is right to break: a default the registry does not describe
gets its token budget from a guess, silently. Splitting these two edits was
tried and abandoned for exactly this reason.

The replacement is chosen for DETECTION, not latency, and that is the part
worth arguing about. On a bait skill carrying credential exfiltration disguised
as a synchronisation step, the fast served model (deepseek-v4-flash-0731,
~1.6 s/call) completed every call, reported no degradation, and returned a
clean verdict. A confident false negative is the worst failure mode a security
scanner has: it is the case where someone signs off. z-ai/glm-5.2 costs ~16 s
per call on the same skill and returns CRITICAL.

The meta_analyzer slot loses its override rather than gaining a new one. The
aggregation pass does benefit from a stronger model, but naming a second model
doubles the surface that can go stale — which is how the previous default
rotted unnoticed. Happy to restore an override if preferred.

Refs #388

Signed-off-by: Mark2Mac <Mark2Mac@users.noreply.github.com>

---------

Signed-off-by: Mark2Mac <Mark2Mac@users.noreply.github.com>
Co-authored-by: Mark2Mac <Mark2Mac@users.noreply.github.com>
Co-authored-by: Narendran Raghavan <32655573+rng1995@users.noreply.github.com>
2026-08-20 23:27:24 +00:00
Marco Macrì 212ebbb011 fix(nv_build): declare glm-5.2's real limits so calls stop failing (#390)
The bundled registry gave z-ai/glm-5.2 a 1000000-token context window and no
output cap. model_info derives the output budget as
ctx * (1 - MAX_INPUT_TOKENS_PCT), so every request asked for 250000 output
tokens and the endpoint answered:

  400 This model configuration accepts at most 202749 combined input and output
      tokens. However, your request has 1249 input tokens and asks for 250000
      output tokens (251249 tokens total).

202749 is quoted verbatim by the endpoint in that 400. With the entry corrected
the same scan completes with 4/4 LLM calls and the meta-analyzer applied.

An over-stated context window does not degrade gracefully: it zeroes the LLM
stage, and nothing in the error points at the registry. Under-stating is safe,
over-stating is not.

Limits may vary per account, which is now noted in the YAML.

Scope is deliberately one entry. The registry also names three models the
catalogue no longer serves, but removing them is coupled to DEFAULT_MODEL by an
invariant the suite already asserts ("nv_build's default model is in its
registry"), so that change travels with the default in a separate PR.

Refs #388

Signed-off-by: Mark2Mac <Mark2Mac@users.noreply.github.com>
Co-authored-by: Mark2Mac <Mark2Mac@users.noreply.github.com>
Co-authored-by: Narendran Raghavan <32655573+rng1995@users.noreply.github.com>
2026-08-20 16:21:21 -07:00
green3sf 71bb4d32b1 fix(lp1): tailor remediation to manifest type (#402)
Signed-off-by: green3sf <222944370+green3sf@users.noreply.github.com>
Co-authored-by: Narendran Raghavan <32655573+rng1995@users.noreply.github.com>
2026-08-20 23:15:18 +00:00
Rod Boev 305479ddac feat(analyzer): add phase-1 structured skill summaries (#211)
* feat(analyzer): add phase-1 structured skill summaries

Signed-off-by: Rod Boev <rod.boev@gmail.com>

* test(analyzer): keep structured workflow summaries stable

Signed-off-by: Rod Boev <rod.boev@gmail.com>

* fix(analyzer): parse issue-shaped AISOP metadata

Signed-off-by: Rod Boev <rod.boev@gmail.com>

* fix(analyzer): fail closed on over-nested structured bundles

Signed-off-by: Rod Boev <rod.boev@gmail.com>

* fix(report): keep structured summaries out of findings (#130)

Signed-off-by: Rod Boev <rod.boev@gmail.com>

* fix(structured-skill): keep discovery root-relative (#130)

Signed-off-by: Rod Boev <rod.boev@gmail.com>

---------

Signed-off-by: Rod Boev <rod.boev@gmail.com>
Signed-off-by: Narendran Raghavan <32655573+rng1995@users.noreply.github.com>
Co-authored-by: Narendran Raghavan <32655573+rng1995@users.noreply.github.com>
2026-08-20 16:11:27 -07:00
Andrew White ab203686aa fix: simplify redundant error-message assertion (#385)
Signed-off-by: andrewwhitecdw <andrewwhitecdw@users.noreply.github.com>
Co-authored-by: andrewwhitecdw <andrewwhitecdw@users.noreply.github.com>
Co-authored-by: Narendran Raghavan <32655573+rng1995@users.noreply.github.com>
2026-08-20 23:01:48 +00:00
Werner Kasselman 33b3930306 fix(cli): report the findings that actually drove the risk score (#375)
* fix(cli): report the findings that actually drove the risk score

Four call sites selected findings with `filtered_findings or findings`,
which over-reports in two distinct ways.

The falsy fallback. `report` returns `filtered_findings` as a real list,
and an empty one is a real answer: every finding was filtered out by the
meta-analyzer, or suppressed by a baseline. The `or` treats `[]` as absent
and falls through to the raw pre-filter `findings`, so a skill that scores
0 is reported with a non-zero finding count.

The unsubtracted partition. `report` returns `filtered_findings` as the
full pre-partition set (kept plus baseline-suppressed) alongside
`suppressed_findings`, and scores, dedupes, and builds SARIF from the kept
subset alone. Counting `filtered_findings` therefore counts findings the
report itself excluded.

Adds `suppression.effective_findings()`, the inverse of the existing
`partition_findings()`, and routes all four sites through it:

- the recursive multi-skill summary table (`cli.py`)
- the combined recursive JSON report (`cli.py`)
- `skillspector baseline`, which previously fingerprinted raw findings
  the scan had already filtered out
- the MCP `scan_skill` verdict, which serialises this list straight to a
  calling agent, so a suppressed finding leaking in tells that agent a
  skill is dirtier than the score it is gating on

It falls back to the raw `findings` list only when `filtered_findings` is
absent or malformed, and does not subtract there, since raw findings are
not the population that produced `suppressed_findings`.

Verified by reverting the source change against the new tests: all five
behavioural tests fail on the current code at exactly the site each one
targets, and pass with the fix. 2088 passed, 17 skipped, 4 xfailed.

Signed-off-by: Werner Kasselman <145896621+wernerkasselman-au@users.noreply.github.com>

* test(suppression): close the mutation survivors in effective_findings

A mutation harness against the shipped suite ran nineteen mutants: thirteen
killed, six survivors. A survivor is an unprotected behaviour even when the
code is correct, and two of these were real coverage holes rather than
defensive noise.

The two that mattered:

- The `skillspector baseline` call site had no site-level test at all.
  Reverting `cli.py` to the old `filtered_findings or findings` left the entire
  suite green, so the changed fingerprinting behaviour was completely
  unprotected. That line was flagged twice.
- `effective_findings` subtracts by `finding_id`, but swapping both comparisons
  to `rule_id` also left the suite green, because no test had a kept and a
  suppressed finding sharing a rule id. Two hits of one rule at different sites
  is the common case, and keying on `rule_id` would drop the finding that was
  never baselined.

The remaining four covered the malformed-result guards: the suppressed
container type check, the `SuppressedFinding` entry check, the
`entry.finding is not None` check, and the filtered-item `Finding` check. The
container check needed a non-iterable value to be observable, since a truthy
non-list string iterates harmlessly and yields the same answer; an int or float
raises TypeError out of the comprehension without the guard.

Seven new tests, no source change. Each was verified to fail against its own
mutant and pass against the restored source, so none of them is green by
accident.

2098 passed, 17 skipped, 4 xfailed. Ruff clean. Mypy unchanged at 117 errors
in 21 files, the same count as base.

Signed-off-by: Werner Kasselman <145896621+wernerkasselman-au@users.noreply.github.com>

---------

Signed-off-by: Werner Kasselman <145896621+wernerkasselman-au@users.noreply.github.com>
Co-authored-by: Narendran Raghavan <32655573+rng1995@users.noreply.github.com>
2026-08-20 22:50:42 +00:00
Ho Geun Choi 64731afd09 fix(analyzers): rely on runner for example filtering in E5 and TM4 (#237)
E5 (#218) and TM4 (#220) called is_code_example() with an unconditional continue, letting a nearby example marker (e.g. "# for example") suppress findings in executable files. The shared runner already filters examples in non-executable docs and only downweights executables, so the analyzer-level call was redundant and created an attacker-controlled bypass — the same issue fixed for SC7 in #224. Remove it from both analyzers; replace TM4's analyze-level doc-exclusion test with an executable-evasion test and add the equivalent E5 regression.

Signed-off-by: CharmingGroot <ohyes9711@gmail.com>
Co-authored-by: Narendran Raghavan <32655573+rng1995@users.noreply.github.com>
2026-08-20 22:48:00 +00:00
Marco Macrì d2f1832fda feat(report): expose max_issue_severity so a normalized verdict cannot hide a HIGH finding (#398)
risk_assessment.severity is a normalized, confidence-weighted verdict and can
read LOW/SAFE while issues[] contains a HIGH finding: a single HIGH scores below
the HIGH band. The smoothing is intentional, but it was invisible, so every
consumer that wanted to gate on the worst finding had to walk issues[] and
re-implement the severity ranking.

max_issue_severity reports the highest severity present in issues[], or NONE
when there are none. Additive: no existing field changes value and no scoring is
touched.

_SEVERITY_RANK is kept separate from _SEVERITY_POINTS on purpose — the latter
are scoring weights that may be retuned, this is an ordering consumers will
depend on. Suppressed findings do not raise the value, since a finding excluded
by a baseline is not a reported issue.

Refs #397

Signed-off-by: Mark2Mac <Mark2Mac@users.noreply.github.com>
Co-authored-by: Mark2Mac <Mark2Mac@users.noreply.github.com>
Co-authored-by: Narendran Raghavan <32655573+rng1995@users.noreply.github.com>
2026-08-20 22:44:05 +00:00
Narendran Raghavan e27a973ca6 ci: update PR branches after main changes (#376)
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
2026-08-20 15:41:09 -07:00
badhope 550b9f00ad build: make langgraph-cli[inmem] an optional dependency (#354)
Signed-off-by: badhope <weed33834@users.noreply.github.com>
2026-08-20 14:13:21 -07:00
Narendran Raghavan 29b0dc8c39 release: SkillSpector 2.9.6 (#395)
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
v2.9.6
2026-08-18 13:41:43 -07:00
Narendran Raghavan 8cb94f8125 fix(pe3): distinguish OAuth token nouns from access actions (#392)
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
2026-08-18 12:46:29 -07:00
Keshav Pradeep 27fd9620db release: SkillSpector 2.9.5 (#378)
Signed-off-by: keshavp <32313895+keshprad@users.noreply.github.com>
v2.9.5
2026-08-15 14:03:10 -07:00
Narendran Raghavan 5680c2c300 Scope the locality guard to the namespace (#365)
* fix(yara): scope locality guard to agent namespace

Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>

* Preserve built-in YARA namespace precedence

Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>

---------

Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
2026-08-14 16:09:57 -07:00
Narendran Raghavan 81ff68831e Use byte offsets for YARA line lookup (#364)
* fix(yara): use byte offsets for line lookup

Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>

* Fix zero-offset YARA match location

Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>

---------

Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
Signed-off-by: Keshav Pradeep <32313895+keshprad@users.noreply.github.com>
Co-authored-by: Keshav Pradeep <32313895+keshprad@users.noreply.github.com>
2026-08-14 18:04:23 -04:00
Rod Boev 66b2890c59 feat(cli): opt-in discovery of an author-shipped baseline (#278) (#286)
* feat(cli): opt-in discovery of an author-shipped baseline (#278)

Discover a co-located .skillspector-baseline.yaml and apply it only when
the consumer opts in with --use-shipped-baseline, reporting provenance on
stderr. Detection without opt-in leaves findings and the risk score
untouched and never parses the file. Explicit --baseline still wins.

Closes #278

Signed-off-by: Rod Boev <rod.boev@gmail.com>

* test: ignore run-unique finding ids in baseline checks

Signed-off-by: Rod Boev <rod.boev@gmail.com>

---------

Signed-off-by: Rod Boev <rod.boev@gmail.com>
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
Co-authored-by: Narendran Raghavan <32655573+rng1995@users.noreply.github.com>
Co-authored-by: Narendran Raghavan <nraghavan@nvidia.com>
2026-08-14 14:10:00 -07:00
Mohammed Imran Khan f2f94b41fd feat: add Ollama, Azure OpenAI, and generic OpenAI-compatible providers (#179)
Ollama provider enables free local/offline LLM scanning via Ollama's
OpenAI-compatible API at localhost:11434. No API key required.

Azure OpenAI provider supports enterprise deployments using
AzureChatOpenAI with deployment-based routing and api-version handling.

Generic OpenAI-compatible provider serves Groq, Together AI, Mistral,
DeepInfra, and other endpoints with dedicated env vars
(SKILLSPECTOR_COMPAT_API_KEY/BASE_URL) and a bundled multi-provider
model registry.

Each provider ships its own model_registry.yaml for accurate token
budgeting.

Closes #173, closes #174, closes #175

Signed-off-by: mimran-khan <mohammed_imran.khan@outlook.com>
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
Co-authored-by: Narendran Raghavan <32655573+rng1995@users.noreply.github.com>
Co-authored-by: Narendran Raghavan <nraghavan@nvidia.com>
2026-08-14 14:06:51 -07:00
Parafee41 06fc723add fix MCP dependency compatibility (#339)
Signed-off-by: kigland <shuaizhicheng336@gmail.com>
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
Co-authored-by: Narendran Raghavan <32655573+rng1995@users.noreply.github.com>
Co-authored-by: Narendran Raghavan <nraghavan@nvidia.com>
2026-08-14 14:01:52 -07:00
Parafee41 aace221e15 fix(analyzers): reduce false positives for negated safety constraints (#254)
* Reduce false positives for negated safety constraints

Signed-off-by: kigland <shuaizhicheng336@gmail.com>

* fix static false-positive guard bypasses

Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>

* tighten static guard edge cases

Signed-off-by: kigland <shuaizhicheng336@gmail.com>

* scope cache cleanup suppression to single commands

Signed-off-by: kigland <shuaizhicheng336@gmail.com>

---------

Signed-off-by: kigland <shuaizhicheng336@gmail.com>
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
Co-authored-by: Narendran Raghavan <32655573+rng1995@users.noreply.github.com>
Co-authored-by: Narendran Raghavan <nraghavan@nvidia.com>
2026-08-14 13:55:50 -07:00
Ram a1fca7ceff feat(analyzer): detect insecure deserialization (AST10, TT6, DS1–DS4) (#246)
* feat(analyzer): detect insecure deserialization (AST10, TT6, DS1-DS4)

Closes the insecure-deserialization gap (OWASP ASI05 - Unexpected Code
Execution) across the analyzer stack:

- behavioral_ast (AST10): flags pickle / marshal / dill / jsonpickle /
  joblib / pandas.read_pickle, plus argument-aware yaml.load, torch.load,
  and numpy.load so the hardened forms (SafeLoader, weights_only=True,
  default allow_pickle=False) are not false-positived.
- behavioral_taint_tracking (TT6): external or file input -> deserialization
  sink, the RCE-class flow analogue of TT5.
- static_patterns_deserialization (DS1-DS4): language-gated regex breadth
  for the non-Python scripts a skill may bundle (PHP unserialize, Ruby
  Marshal/YAML/Oj, JS node-serialize/funcster).

Registers the new analyzer node, adds rule metadata (explanations,
remediations, category, pattern names), and ships unit tests for all rules
including hardened-form and language-gating negative cases.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Ram Dwivedi <abhiram.dwivedi@yahoo.com>

* fix(analyzer): catch positional allow_pickle in numpy.load (AST10)

numpy.load(file, mmap_mode, allow_pickle) accepts allow_pickle as its
third positional argument; AST10 only checked the allow_pickle= keyword,
so numpy.load(f, None, True) bypassed the deserialization finding.

Signed-off-by: Ram Dwivedi <abhiram.dwivedi@yahoo.com>

* fix(analyzer): emit inspection-ledger events from the deserialization node

The DS1-DS4 node was written before main's inspection-ledger refactor and
still called run_static_patterns, so it was the only static analyzer that
reported findings without accounting for the files it inspected: it never
appeared in analysis_completeness or the analyzer-status table.

Switch to run_static_patterns_with_ledger, matching the other 14 static
analyzers, and cover the completed/skipped work items with tests.

Signed-off-by: Ram Dwivedi <abhiram.dwivedi@yahoo.com>

---------

Signed-off-by: Ram Dwivedi <abhiram.dwivedi@yahoo.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-14 13:37:03 -07:00
Keshav Pradeep 2d198ab910 release: SkillSpector 2.9.4 (#369)
Signed-off-by: keshavp <32313895+keshprad@users.noreply.github.com>
v2.9.4
2026-08-12 23:25:10 -07:00
Rod Boev f883df60ef fix(mcp): reject local targets over HTTP transport (#196)
* fix(mcp): reject local targets over HTTP transport

Signed-off-by: Rod Boev <rod.boev@gmail.com>

* fix(mcp): close direct builder local-target bypass

Signed-off-by: Rod Boev <rod.boev@gmail.com>

* fix(mcp): keep target policy fail closed across paths

Signed-off-by: Rod Boev <rod.boev@gmail.com>

---------

Signed-off-by: Rod Boev <rod.boev@gmail.com>
2026-08-12 03:19:09 -07:00
青衫 64c6299931 Add Skill Inspector companion skill (#253)
Signed-off-by: 青衫 <dx.boy266@gmail.com>
2026-08-12 03:12:33 -07:00
MinYi Xie 2b408ee64d fix(lp3): remediation and docs name allowed-tools for SKILL.md (#316)
LP3's detection logic accepts `allowed-tools` as a valid tool-scope
declaration, but the runtime remediation string, the pattern-defaults
fallback, and docs/B.3.1 still directed authors to add a `permissions`
field. For Claude Code / Agent Skills SKILL.md, `permissions` is not part
of the frontmatter schema and is ignored as unknown, so following the
advice could never resolve the finding.

Aligned the three user-facing strings with the code:

- LP3 remediation (analyzer + pattern_defaults): declare `allowed-tools`
  in SKILL.md frontmatter, or a `permissions` list in MCP server
  manifests, stating which applies to which manifest type.
- LP3 finding message + description: "declares no tool scope
  ('permissions' or 'allowed-tools')" instead of "no declared
  permissions", matching the actual trigger condition.
- docs/B.3.1 LP3 section: Triggers when / Example / Remediation updated
  the same way.

No behavior change; detection logic untouched. tests/test_mcp_least_privilege.py 15/15.

Closes #313

Signed-off-by: ppcvote <risky9763@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-12 02:53:38 -07:00