browser-use--browser-harness
4.2 KiB
4.2 KiB
AGENTS.md
For agents modifying bu. For using it, see SKILL.md.
Philosophy
Both agent-browser (60+ verbs) and browser-use (~20 verbs) are walled gardens. bu inverts this: few helpers, LLM edits them at runtime. Every design decision flows from that.
Architecture
Chrome / Browser Use cloud ── CDP WS ─▶ daemon.py ── /tmp/bu-<NAME>.sock ─▶ run.py
Protocol: one JSON line per direction. Request: {method, params, session_id} (CDP passthrough) or {meta: ...}. Response: {result} / {error} / {events} / {session_id}.
Daemon attaches to the first real page at startup, buffers events in deque(maxlen=500).
BU_NAME (default default) suffixes socket/pid/log — multiple daemons coexist, no supervisor. BU_CDP_WS overrides the local DevToolsActivePort lookup (remote via start_remote_daemon()). BU_BROWSER_ID + BROWSER_USE_API_KEY → daemon stops the cloud browser on shutdown.
Design decisions worth preserving
- Coordinate clicks default.
Input.dispatchMouseEventgoes through iframes/shadow/cross-origin at the compositor level — no per-frame session juggling. - Connect to user's running Chrome, never launch one. User sees what's happening; cookies/logins are theirs.
- cdp-use is used only for
CDPClient.send_raw. The 36k lines of typed wrappers are IDE sugar; raw CDP strings tokenize better and the LLM knows method names from training. run.pyis 3 lines on purpose. No argparse, no subcommands. LLM writes Python.- helpers.py is editable at runtime. This is the whole point.
Rules when extending
- Helpers ≤ 15 lines. No classes. No deps beyond stdlib + cdp-use + websockets.
- Don't add meta verbs lightly; if it can be a helper calling
cdp(), it's a helper. - Never add: CLI argparse, tests, logging framework, config files, session manager, retries, daemon supervisor. Multiple daemons are fine (one per
BU_NAME) — just don't build a thing that manages them. - Taste test: could the LLM rewrite this from scratch after reading it once?
Known gotchas
- Chrome 144+
chrome://inspect/#remote-debuggingdoes NOT serve/json/version. Daemon reads<ChromeProfile>/DevToolsActivePortinstead. Don't suggest the user launch with--remote-debugging-port— they don't want that. - Omnibox popups are
type: "page"CDP targets with ~50px viewports. Filter by URL prefix (is_real_pagein daemon.py,INTERNALtuple shared with helpers.py). - CDP target order != Chrome's visible tab-strip order. Use UI automation when the user means "the first/second tab I can see".
switch_tab()re-attaches only;Target.activateTargetshows it. type_in/clear uses Cmd+A (macOS). Linux/Windows:2instead of4for modifiers.send_rawhas no timeout — stuck call hangs forever. Add a wrapper if it bites.- Daemon's default session goes stale if user closes the attached tab manually.
ensure_real_tab()re-attaches. - Two tuples named
INTERNAL(daemon.py, helpers.py) — cross-process, can't share module. Keep in sync. - Browser Use API is camelCase on the wire (
cdpUrl,proxyCountryCode). The SDKs rename — we don't. - Remote
cdpUrlis HTTPS, not ws.cdp_ws_from_url()hits/json/version→webSocketDebuggerUrl. - Stop a cloud browser with
PATCH /browsers/{id}{"action":"stop"}.POST /sessions/{id}/stopis for agent sessions.
Session lessons
- Half the original helpers were never called in practice. Dropped:
get_dom,element_pos,click_element,type_in,save_cookies,load_cookies,set_viewport,screenshot_full,double_click,right_click,move_mouse,new_tab,close_tab,handle_dialog,back,reload. Every DOM interaction went throughjs("...")with a bespoke selector. http_get+ThreadPoolExecutorbeats the browser for static scrapes. 249 Netflix pages in 2.8s parallel.wait(5)after goto is fragile.wait_for_load()pollsdocument.readyState.- Auth-gated sites (Upwork, X) redirect to login. Not our problem; bail and ask the user.
- Screenshots render at ~half viewport width in the transcript. Don't eyeball click coords off the image — use
js("el.getBoundingClientRect()")for the real pixel location.