Refs #2052
This change updates all the eval scenarios so that they work with and
without experimentalPageIdRouting flag
Co-authored-by: Samiya Caur <samiyac@chromium.org>
Improve fill_form eval by checking if fill_form was the only tool called
for the task and provide more descriptive failure messages.
Co-authored-by: Piotr Paulski <piotrpaulski@chromium.org>
This eval represents minimal reproduction scenario for #1942. Currently
fill_form is used in around 25% of the attempts.
It can also be used to reproduce #1941 with about 1:10 chance.
---------
Co-authored-by: Piotr Paulski <piotrpaulski@chromium.org>
Addresses cases where DevTools MCP tools were not consistently picked up
from natural language prompts by improving tool descriptions and
metadata.
Validation:
Tested locally across multiple prompts related to LCP and page
performance.
MCP tools were selected more consistently after the description updates.
Refs #940
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Alex Rudenko <OrKoN@users.noreply.github.com>
Co-authored-by: Nikolay Vitkov <34244704+Lightning00Blade@users.noreply.github.com>
Co-authored-by: Alex Rudenko <alexrudenko@chromium.org>
## Summary
Adds optional `pageId` routing to page-scoped tools, gated behind
`--experimental-page-id-routing`. When enabled, multi-agent callers can
target a specific page without relying on global selection state. Fully
backward-compatible: without the flag, behavior is unchanged.
### Key changes
- **`pageScoped` annotation**: tools declare `pageScoped: true`; the
server merges `pageId` into their schema at registration time (when the
flag is on)
- **`McpPage` wrapper**: consolidates per-page state (numeric id,
isolated context name, focus tracking) into a single class
- **Request-scoped page routing**: `resolvePageById()` resolves the
target page, `setRequestPage()` threads it through the handler so tools
like `getSelectedPage()` see the right page
- **`assertPageIsFocused`**: keyboard/input tools validate that the
target page holds browser focus, returning an actionable error ("call
select_page first") instead of silently dispatching to the wrong page
- **`--experimental-page-id-routing` CLI flag** (hidden): gates schema
injection and request-scoped routing so the feature can be tested before
graduating
- **Eval scenarios**: `page_id_routing_test` and
`page_focus_keyboard_test` with `serverArgs` support in the eval harness
Addresses #1019
## Summary
Adds storage-isolated browser contexts via an optional `isolatedContext`
parameter on the `new_page` tool, following the simplified design
proposed by @OrKoN in #926.
Pages created with the same `isolatedContext` name share cookies,
localStorage, and storage. Pages in different isolated contexts (or the
default context) are fully isolated — ideal for testing multi-user
real-time features like chat, notifications, or collaborative editing.
## Changes
### `new_page` tool
- New optional `isolatedContext: string` parameter
- If specified, creates/reuses a named `BrowserContext` and opens a page
in it
- If omitted, uses the default browser context (existing behavior
unchanged)
### `McpContext`
- `#isolatedContexts` Map: LLM-provided names → Puppeteer
`BrowserContext` instances
- `#pageToIsolatedContextName` WeakMap: GC-safe page → context name
reverse lookup
- Auto-discovery: externally created browser contexts get
`isolated-context-1`, `isolated-context-2`, etc.
- `getIsolatedContextName(page)`: returns the isolated context name for
a page (used by response formatting)
- `page.browserContext()` used for context membership detection (no
custom target event forwarding needed)
- No context cleanup in `dispose()` or `closePage()` — either the entire
browser is closed or we disconnect without destroying state
### `McpResponse`
- Page list includes `isolatedContext=${name}` labels (both text and
structured JSON output)
### `ToolDefinition`
- `Context` interface extended with `getIsolatedContextName(page)`
method
## What's NOT included (by design)
- **No `TargetEventEmitter`**: Puppeteer forwards target events from
`BrowserContext` → `Browser` internally
- **No context cleanup**: Browser contexts are not closed on `dispose()`
or page close, per maintainer guidance
- **No `about:blank` cleanup**: Default context and isolated contexts
coexist side-by-side
## Example
```
> new_page url="https://app.example.com/chat" isolatedContext="userA"
> new_page url="https://app.example.com/chat" isolatedContext="userB"
> list_pages
Page 1: [app.example.com/chat] isolatedContext=userA
Page 2: [app.example.com/chat] isolatedContext=userB [selected]
```
Pages in different isolated contexts have fully independent cookies,
localStorage, IndexedDB, and WebSocket connections.
## Tests
- 6 new tests covering `isolatedContext` feature in
`tests/tools/pages.test.ts`
- All existing tests pass (333+)
- Zero type errors, lint clean
Closes#926
---------
Co-authored-by: Alex Rudenko <alexrudenko@chromium.org>
We should make sure there is always an empty line after license notice,
otherwise it gets dropped during build if it happens to be before import
statement.
---------
Co-authored-by: Piotr Paulski <piotrpaulski@chromium.org>
This PR adds a tool based on the node test runner that runs a loop to
see what tools a model chooses given a prompt. The expectations are
encoding for each prompt. Run `npm run eval` to get results. Currently,
only Gemini and needs and API key.
Samiya Caur