Removes `--experimental-page-id-routing` making it the default behavior.
To go back to the previous behavior, pass `--pageIdRouting=false` when
starting the server.
---------
Co-authored-by: Alex Rudenko <alexrudenko@chromium.org>
Co-authored-by: Samiya Caur <samiyac@google.com>
To see the heap size increase the scenario has to be run with
warmup-iterations=0. I have made the number of iterations different by
scenario, unless they are overriden through the command line.
Once we have the fixes all merged, I will add an assert somewhere and
see if this can be added into the test suite.
This is mostly a refactor except:
- the url param is removed. The source map server is always turned on.
We can always manually connect the mcp server to an external site and
check for memory usage / leaks. Testing should be deterministic and not
rely on outside resources.
- the reports are now stored by scenario. Console still prints out
report for each scenario, alongside a final report giving a one line
summary per scenario.
Still a bit rough, but useful.
Adding devtools-frontend from github as a submodule with depth of 1 to
speed up checkouts. prepare.ts script should automatically pull the
right version.
Fixes#955
## Summary
- update ESLint and `@eslint/js` to v10
- replace the import lint plugin with an ESLint 10-compatible
implementation while preserving the existing import and Node protocol
checks
- update the local license rule for ESLint 10's rule-context properties
- align the supported Node.js versions with ESLint 10's engine
requirements
- preserve the current recommended-rule baseline during the major
upgrade
## Testing
- `npm ci`
- `npm run check-format`
- `npm run typecheck`
- `npm run test`
Refs #2052
This change updates all the eval scenarios so that they work with and
without experimentalPageIdRouting flag
Co-authored-by: Samiya Caur <samiyac@chromium.org>
This also caught and fixed an issue where the enum index is not
sequential :/
No error enums have been defined on the service side yet so it's
possible to modify the enum index directly.
There is no behavior change.
This groups utilities into two modules:
- `transformation.ts`: for any logic related to mutating / filtering of
the names, values of telemetry entries.
- `metricsRegistry.ts`: for any logic that related to the maintenance of
metrics.json files.
Improve fill_form eval by checking if fill_form was the only tool called
for the task and provide more descriptive failure messages.
Co-authored-by: Piotr Paulski <piotrpaulski@chromium.org>
Extracting WebMCP tools into a separate category for better grouping in
the docs. This changes `--experimentalWebmcp` to
`--categoryExperimentalWebmcp` to align with other experimental
categories. Debugging category was not a good fit since the tools
provided by WebMCP are not necessarily used for debugging.
cc @beaufortfrancois
This eval represents minimal reproduction scenario for #1942. Currently
fill_form is used in around 25% of the attempts.
It can also be used to reproduce #1941 with about 1:10 chance.
---------
Co-authored-by: Piotr Paulski <piotrpaulski@chromium.org>
This is the second most highly touched metric areas - so let's automate
this portion first. The approach is similar to what we have for tool
call args. The append-only logic will be added in the follow-up PR:
#1882
Under some cases one could not re-build correctly due to the directory
being remove.
And we can save some CPU cycles in test to disable the update checker.
the node test framework runs not only tests respecting
`--test-concurrency=1` but also discovery of the tests (import of the
files). This makes the `npm run test:only` slow. This PR checks ahead of
times which files have .only and only tells the node test runner to
import those.
With this change new tools are added to the very back of the json file.
Any removed tools will receive a isDeprecated flag in the existing
entry. And the same with tool arguments.
This adds a script that generates a json file that summarizes all tool
calls and arguments for each.
- The arguments run through the blocklist filtering so arguments
containing high entropy ids are filtered out (e.g. "uid", "msgid" etc).
- It uses existing functions from clearcut logger module to transform
the arg name and value (i.e. take the length of the string, take the
size of the array, and rename the string to be "string_length", and
array "array_count" etc).
- These functions from the clearcut logger module will be later used to
sanitize the params as we start to log them.
This doesn't include the append only / deprecation logic just yet (i.e.
it's doesn't handle the case when new tools are added / removed, or
arguments of the existing tools are modified). This will be added in
following PRs.
The parent PR is #1250.
## Summary
- Adds a custom ESLint rule `@local/no-direct-third-party-imports` that
flags value imports of bundled third-party packages
(`@modelcontextprotocol/sdk`, `puppeteer-core`, `@puppeteer/browsers`,
`yargs`, `debug`, `zod`, `core-js`) when used outside of
`src/third_party/`
- Type-only imports (`import type`) are allowed since they are erased at
compile time and don't affect the bundle
- The rule is scoped to `src/**/*.ts` so development scripts and tests
are unaffected
This prevents the class of bugs where a direct npm import works during
development (devDependencies installed) but breaks in the published
package (only bundled code ships). PR #1111 was an example of this exact
issue caught through manual `npm pack` testing.
Closes#1123
## Test plan
- [x] Verified `npx eslint --no-cache src/` passes with no violations on
the current codebase
- [x] Verified the rule correctly catches a test file with `import
{Client} from '@modelcontextprotocol/sdk/client/index.js'`
- [x] Verified the rule allows `import type {Flags} from 'lighthouse'`
(type-only import)
- [x] Verified the rule does not fire inside `src/third_party/index.ts`
(the barrel itself)
- [x] Verified scripts/ and tests/ are unaffected (rule scoped to
`src/**/*.ts`)
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Alex Rudenko <alexrudenko@chromium.org>
## Summary
- Add `@local/no-zod-nullable-object` ESLint rule that disallows
`.nullable()` and `.object()` usage in tool schema files
(`src/tools/**/*.ts`)
- Fix the existing `zod.object()` violation in `fill_form` by converting
the elements array from objects to `"uid=value"` formatted strings
- Remove the TODO from CONTRIBUTING.md and link the rule name to the
documented restriction
## Context
Per the [PR #1073
review](https://github.com/ChromeDevTools/chrome-devtools-mcp/pull/1073#discussion_r2872188785)
and the CONTRIBUTING.md guidelines, tool schemas should not use
`.nullable()` or `.object()` types. Complex objects should be
represented as short formatted strings instead.
The rule catches:
- `zod.object({...})` / `z.object({...})` - flags direct zod object
schema usage
- `.nullable()` - flags nullable schema usage on any expression
Closes#1076
## Test plan
- [ ] `npx eslint src/tools/` passes with no violations
- [ ] Creating a test file with `zod.object()` or `.nullable()` in
`src/tools/` triggers the rule
- [ ] TypeScript compilation passes (`npx tsc --noEmit`)
- [ ] Existing `fill_form` test updated to use new string format
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Alex Rudenko <OrKoN@users.noreply.github.com>
Co-authored-by: Nikolay Vitkov <34244704+Lightning00Blade@users.noreply.github.com>
Co-authored-by: Alex Rudenko <alexrudenko@chromium.org>
- `chrome-devtools-mcp.js` is the `npx chrome-devtools-mcp`
- `chrome-devtools.js` is the new CLI
- `-cli-options.js` is the corresponding options
- all these files are in the bin folder to indicate they are executable
Addresses cases where DevTools MCP tools were not consistently picked up
from natural language prompts by improving tool descriptions and
metadata.
Validation:
Tested locally across multiple prompts related to LCP and page
performance.
MCP tools were selected more consistently after the description updates.
Refs #940
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Alex Rudenko <OrKoN@users.noreply.github.com>
Co-authored-by: Nikolay Vitkov <34244704+Lightning00Blade@users.noreply.github.com>
Co-authored-by: Alex Rudenko <alexrudenko@chromium.org>
Tolgahan Demirbaş