Improvements for handling in-page tool responses. In order to
successfully pass an in-page tool response from the page context to the
MCP server, the response needs to be serializable. The code walks the
response object an performs the following changes:
- DOM elements are stashed onto the window object and replaced with an
ID. On the MCP server side this ID is used to map back to the
corresponding UID in the page snapshot generated from the accessibility
tree.
- Circular references are replaced with a string.
- Class instances (which can be complex or non-serializable) are
replaced with a string.
- Functions are replaced with a string.
If the in-page tool response contains DOM elements which are not part of
the page snapshot, a new snapshot is created add the missing elements
are added explicitly.
DOM elements are non-serializable and therefore cannot be directly sent
between the inspected page and the MCP server. JSONSchema also has no
native type for DOM elements.
If an in-page tool expects a DOM element as an input parameter, it
should specify this in its input schema by adding `'x-mcp-type':
'HTMLElement'` to the object it expects to be a DOM element.
The MCP server internally refers to DOM elements by a UID (UIDs are
assigned when generating a page snapshot which is based on the page's
accessibility tree).
This change provides the mapping between DOM element and UID in both
directions:
1) The tool's input schema is rewritten internally, adding a required
UID attribute to objects with `'x-mcp-type': 'HTMLElement'`. This allows
the MCP server to call the in-page tool with UIDs where the tool expects
DOM elements.
2) In the page context, the UIDs are replaced with the corresponding DOM
elements, before the actual in-page tool is called. This means that the
in-page tool receives DOM elements as parameters where it expects them.
This moves the stored `ToolGroup` from `McpContext` to `McpPage`, where
per-page state should be stored in order for pageId-based routing to
work correctly.
When appending the list of in-page tools to a response, the list now
corresponds to the response's `McpPage`, and only falls back to the
selected `McpPage` for tools which are not page-specific.
---------
Co-authored-by: browser-automation-bot <133232582+browser-automation-bot@users.noreply.github.com>
Co-authored-by: Alex Rudenko <alexrudenko@chromium.org>
Co-authored-by: Tolgahan Demirbaş <49946947+bcfmtolgahan@users.noreply.github.com>
This allows the MCP server to call the in-page tools provided by the
inspected page.
Handling of (non-serializable) DOM elements as tool parameters or tool
output will be added in follow-ups.
There is no mechanism which would allow a page to push an updated list
of in-page tools to the MCP server. The next best thing I can think of
is to append an updated list of in-page tools to the response for each
tool call related to page navigation (navigate_page, list_pages,
select_page, close_page, new_page).
This adds a `list_in_page_tools` MCP tool. When called, it dispatches a
`devtoolstooldiscovery` event on the active page. The page announces its
exposing tools by calling the event's `respondWith` method, which causes
the exposed tools to be stashed on the page's `window` object. This list
of in-page tools is then appended to the `list_in_page_tools` response.
Calling the exposed in-page tools from the MCP server will be handled in
a follow-up.
Nicholas Roscino