Refs #2052
This change updates all the eval scenarios so that they work with and
without experimentalPageIdRouting flag
Co-authored-by: Samiya Caur <samiyac@chromium.org>
This also caught and fixed an issue where the enum index is not
sequential :/
No error enums have been defined on the service side yet so it's
possible to modify the enum index directly.
There is no behavior change.
This groups utilities into two modules:
- `transformation.ts`: for any logic related to mutating / filtering of
the names, values of telemetry entries.
- `metricsRegistry.ts`: for any logic that related to the maintenance of
metrics.json files.
Improve fill_form eval by checking if fill_form was the only tool called
for the task and provide more descriptive failure messages.
Co-authored-by: Piotr Paulski <piotrpaulski@chromium.org>
Extracting WebMCP tools into a separate category for better grouping in
the docs. This changes `--experimentalWebmcp` to
`--categoryExperimentalWebmcp` to align with other experimental
categories. Debugging category was not a good fit since the tools
provided by WebMCP are not necessarily used for debugging.
cc @beaufortfrancois
This eval represents minimal reproduction scenario for #1942. Currently
fill_form is used in around 25% of the attempts.
It can also be used to reproduce #1941 with about 1:10 chance.
---------
Co-authored-by: Piotr Paulski <piotrpaulski@chromium.org>
This is the second most highly touched metric areas - so let's automate
this portion first. The approach is similar to what we have for tool
call args. The append-only logic will be added in the follow-up PR:
#1882
Under some cases one could not re-build correctly due to the directory
being remove.
And we can save some CPU cycles in test to disable the update checker.
the node test framework runs not only tests respecting
`--test-concurrency=1` but also discovery of the tests (import of the
files). This makes the `npm run test:only` slow. This PR checks ahead of
times which files have .only and only tells the node test runner to
import those.
With this change new tools are added to the very back of the json file.
Any removed tools will receive a isDeprecated flag in the existing
entry. And the same with tool arguments.
This adds a script that generates a json file that summarizes all tool
calls and arguments for each.
- The arguments run through the blocklist filtering so arguments
containing high entropy ids are filtered out (e.g. "uid", "msgid" etc).
- It uses existing functions from clearcut logger module to transform
the arg name and value (i.e. take the length of the string, take the
size of the array, and rename the string to be "string_length", and
array "array_count" etc).
- These functions from the clearcut logger module will be later used to
sanitize the params as we start to log them.
This doesn't include the append only / deprecation logic just yet (i.e.
it's doesn't handle the case when new tools are added / removed, or
arguments of the existing tools are modified). This will be added in
following PRs.
The parent PR is #1250.
## Summary
- Adds a custom ESLint rule `@local/no-direct-third-party-imports` that
flags value imports of bundled third-party packages
(`@modelcontextprotocol/sdk`, `puppeteer-core`, `@puppeteer/browsers`,
`yargs`, `debug`, `zod`, `core-js`) when used outside of
`src/third_party/`
- Type-only imports (`import type`) are allowed since they are erased at
compile time and don't affect the bundle
- The rule is scoped to `src/**/*.ts` so development scripts and tests
are unaffected
This prevents the class of bugs where a direct npm import works during
development (devDependencies installed) but breaks in the published
package (only bundled code ships). PR #1111 was an example of this exact
issue caught through manual `npm pack` testing.
Closes#1123
## Test plan
- [x] Verified `npx eslint --no-cache src/` passes with no violations on
the current codebase
- [x] Verified the rule correctly catches a test file with `import
{Client} from '@modelcontextprotocol/sdk/client/index.js'`
- [x] Verified the rule allows `import type {Flags} from 'lighthouse'`
(type-only import)
- [x] Verified the rule does not fire inside `src/third_party/index.ts`
(the barrel itself)
- [x] Verified scripts/ and tests/ are unaffected (rule scoped to
`src/**/*.ts`)
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Alex Rudenko <alexrudenko@chromium.org>
## Summary
- Add `@local/no-zod-nullable-object` ESLint rule that disallows
`.nullable()` and `.object()` usage in tool schema files
(`src/tools/**/*.ts`)
- Fix the existing `zod.object()` violation in `fill_form` by converting
the elements array from objects to `"uid=value"` formatted strings
- Remove the TODO from CONTRIBUTING.md and link the rule name to the
documented restriction
## Context
Per the [PR #1073
review](https://github.com/ChromeDevTools/chrome-devtools-mcp/pull/1073#discussion_r2872188785)
and the CONTRIBUTING.md guidelines, tool schemas should not use
`.nullable()` or `.object()` types. Complex objects should be
represented as short formatted strings instead.
The rule catches:
- `zod.object({...})` / `z.object({...})` - flags direct zod object
schema usage
- `.nullable()` - flags nullable schema usage on any expression
Closes#1076
## Test plan
- [ ] `npx eslint src/tools/` passes with no violations
- [ ] Creating a test file with `zod.object()` or `.nullable()` in
`src/tools/` triggers the rule
- [ ] TypeScript compilation passes (`npx tsc --noEmit`)
- [ ] Existing `fill_form` test updated to use new string format
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Alex Rudenko <OrKoN@users.noreply.github.com>
Co-authored-by: Nikolay Vitkov <34244704+Lightning00Blade@users.noreply.github.com>
Co-authored-by: Alex Rudenko <alexrudenko@chromium.org>
- `chrome-devtools-mcp.js` is the `npx chrome-devtools-mcp`
- `chrome-devtools.js` is the new CLI
- `-cli-options.js` is the corresponding options
- all these files are in the bin folder to indicate they are executable
Addresses cases where DevTools MCP tools were not consistently picked up
from natural language prompts by improving tool descriptions and
metadata.
Validation:
Tested locally across multiple prompts related to LCP and page
performance.
MCP tools were selected more consistently after the description updates.
Refs #940
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Alex Rudenko <OrKoN@users.noreply.github.com>
Co-authored-by: Nikolay Vitkov <34244704+Lightning00Blade@users.noreply.github.com>
Co-authored-by: Alex Rudenko <alexrudenko@chromium.org>
The script will be used to generate CLI for the MCP server at
`src/bin/cliDefinitions.ts`.
It uses the live server JSON schemas to make it more compatible with the
actual MCP server at runtime. The code definitions are used to extend
the CLI with tool categories.
## Summary
Adds optional `pageId` routing to page-scoped tools, gated behind
`--experimental-page-id-routing`. When enabled, multi-agent callers can
target a specific page without relying on global selection state. Fully
backward-compatible: without the flag, behavior is unchanged.
### Key changes
- **`pageScoped` annotation**: tools declare `pageScoped: true`; the
server merges `pageId` into their schema at registration time (when the
flag is on)
- **`McpPage` wrapper**: consolidates per-page state (numeric id,
isolated context name, focus tracking) into a single class
- **Request-scoped page routing**: `resolvePageById()` resolves the
target page, `setRequestPage()` threads it through the handler so tools
like `getSelectedPage()` see the right page
- **`assertPageIsFocused`**: keyboard/input tools validate that the
target page holds browser focus, returning an actionable error ("call
select_page first") instead of silently dispatching to the wrong page
- **`--experimental-page-id-routing` CLI flag** (hidden): gates schema
injection and request-scoped routing so the feature can be tested before
graduating
- **Eval scenarios**: `page_id_routing_test` and
`page_focus_keyboard_test` with `serverArgs` support in the eval harness
Addresses #1019
Fixes#1029
## Summary
- Use `pathToFileURL()` from `node:url` for dynamic imports in
`eval_gemini.ts` to fix `ERR_UNSUPPORTED_ESM_URL_SCHEME` on Windows -
converts absolute paths like `D:\projects\...\console_test.ts` to proper
`file:///D:/projects/.../console_test.ts` URLs that the Node.js ESM
loader accepts on all platforms
- Move `CHROME_DEVTOOLS_MCP_NO_USAGE_STATISTICS=true` from Unix-only
shell env var prefix in `package.json` into `eval_gemini.ts` code,
matching the pattern already used in `scripts/test.mjs`
- Add `.gitattributes` with `* text=auto eol=lf` to enforce LF line
endings on checkout - Windows contributors cloning the repo would
otherwise get CRLF endings, which conflicts with the project's Prettier
`endOfLine: 'lf'` config and causes spurious diffs or formatting
failures
## Summary
Adds storage-isolated browser contexts via an optional `isolatedContext`
parameter on the `new_page` tool, following the simplified design
proposed by @OrKoN in #926.
Pages created with the same `isolatedContext` name share cookies,
localStorage, and storage. Pages in different isolated contexts (or the
default context) are fully isolated — ideal for testing multi-user
real-time features like chat, notifications, or collaborative editing.
## Changes
### `new_page` tool
- New optional `isolatedContext: string` parameter
- If specified, creates/reuses a named `BrowserContext` and opens a page
in it
- If omitted, uses the default browser context (existing behavior
unchanged)
### `McpContext`
- `#isolatedContexts` Map: LLM-provided names → Puppeteer
`BrowserContext` instances
- `#pageToIsolatedContextName` WeakMap: GC-safe page → context name
reverse lookup
- Auto-discovery: externally created browser contexts get
`isolated-context-1`, `isolated-context-2`, etc.
- `getIsolatedContextName(page)`: returns the isolated context name for
a page (used by response formatting)
- `page.browserContext()` used for context membership detection (no
custom target event forwarding needed)
- No context cleanup in `dispose()` or `closePage()` — either the entire
browser is closed or we disconnect without destroying state
### `McpResponse`
- Page list includes `isolatedContext=${name}` labels (both text and
structured JSON output)
### `ToolDefinition`
- `Context` interface extended with `getIsolatedContextName(page)`
method
## What's NOT included (by design)
- **No `TargetEventEmitter`**: Puppeteer forwards target events from
`BrowserContext` → `Browser` internally
- **No context cleanup**: Browser contexts are not closed on `dispose()`
or page close, per maintainer guidance
- **No `about:blank` cleanup**: Default context and isolated contexts
coexist side-by-side
## Example
```
> new_page url="https://app.example.com/chat" isolatedContext="userA"
> new_page url="https://app.example.com/chat" isolatedContext="userB"
> list_pages
Page 1: [app.example.com/chat] isolatedContext=userA
Page 2: [app.example.com/chat] isolatedContext=userB [selected]
```
Pages in different isolated contexts have fully independent cookies,
localStorage, IndexedDB, and WebSocket connections.
## Tests
- 6 new tests covering `isolatedContext` feature in
`tests/tools/pages.test.ts`
- All existing tests pass (333+)
- Zero type errors, lint clean
Closes#926
---------
Co-authored-by: Alex Rudenko <alexrudenko@chromium.org>
Alex Rudenko