current_tab() and list_tabs() both return dicts shaped {"targetId": ...,
"url": ..., "title": ...}, but switch_tab() only accepted the bare string.
This made the natural pattern
original = current_tab()
new_tab(...)
switch_tab(original)
raise InvalidParameters from the CDP layer ("string value expected at
position 17"). Callers had to remember to pull ["targetId"] out manually,
which contradicts the harness's "obvious shapes work" ergonomic.
Make switch_tab tolerant of either shape: pull targetId from the dict if
one is passed, otherwise treat the input as the id string.
* fix(js): don't double-wrap IIFEs that contain return
The substring check `"return " in expression` incorrectly matched
expressions that already contained an IIFE with an internal return,
causing double-wrapping and a silent None result. Guard with
`not expression.strip().startswith("(")` so pre-wrapped expressions
pass through unchanged.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(cli): add --reload flag to restart the daemon
Kills the running daemon so the next call picks up code changes,
replacing the manual pkill workflow after editing helpers.py.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
The substring check `"return " in expression` incorrectly matched
expressions that already contained an IIFE with an internal return,
causing double-wrapping and a silent None result. Guard with
`not expression.strip().startswith("(")` so pre-wrapped expressions
pass through unchanged.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
When falling through to the browser path for watch-page DOM (instead of
the http_get + ytInitialPlayerResponse blob), wait_for_load() is not
enough. The load event fires before YouTube's Polymer components
hydrate — h1.ytd-watch-metadata yt-formatted-string,
ytd-video-owner-renderer #channel-name a, and ytd-watch-info-text all
return null for ~2s after load. A wait(3) after wait_for_load() is
required before querying any watch-page selector.
Field-tested 2026-04-24 on Brave; same behavior observed on
ungoogled-chromium. The HTTP path remains the recommended approach for
metadata; this note exists for the cases that genuinely need the
rendered DOM (live UI state, etc.).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Markdown-style domain skill for Polymarket scraping via Gamma API
(api-first per repo doctrine) with DOM leaf-div-disambiguation fallback
for CSS-module SPAs. Covers market outcomes, metadata, and comments.
Live-tested against gamma-api.polymarket.com and a live event page:
- 9 outcomes extracted (e.g. April 7: YES 99.95 / NO 0.05, vol $45.7M)
- Metadata: title, end_date, total_volume, category, market_count
- 38 comments fetched (40 raw, 2 deleted skipped)
Gotcha documented: Gamma API comment envelopes for deleted comments
preserve id/createdAt/profile/media/parentCommentID but drop the body
field entirely — naive dict access throws KeyError. Guard with
'if "body" not in c: continue'.
DOM fallback pattern documented (not primary path): Polymarket has
zero data-testid attributes and CSS-module-hashed classes. Leaf-div
disambiguation (children.length === 0 + nearest-common-ancestor
grouping) is the only robust approach.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When BH_DEBUG_CLICKS=1 (or --debug-clicks CLI flag), click_at_xy
captures a screenshot before each click and draws a red crosshair
at the exact click location, scaled by window.devicePixelRatio so
the marker aligns correctly on HiDPI/Retina displays.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(js): auto-wrap top-level return expressions in an IIFE
Agents naturally write js() calls with top-level `return`, which is a
syntax error in Runtime.evaluate. js() now detects `return ` and wraps
the expression in an IIFE so both styles work without callers needing
to know the CDP constraint.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor(run): replace heredoc stdin with -c flag
Drops the <<'PY' heredoc pattern entirely. Agents now pass code via
`browser-harness -c "..."`, which is simpler to construct programmatically
and avoids heredoc quoting pitfalls.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Agents naturally write js() calls with top-level `return`, which is a
syntax error in Runtime.evaluate. js() now detects `return ` and wraps
the expression in an IIFE so both styles work without callers needing
to know the CDP constraint.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
SKILL.md now covers day-to-day usage only. Maintenance commands
(--doctor, --setup, --update) and the architecture section move to
install.md where setup and break-fix content belongs.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
- goto() → goto_url()
- click() → click_at_xy()
- screenshot() → capture_screenshot()
These three shared exact names with Playwright but different argument
shapes, causing agent confusion. Updated helpers.py and all markdown
skill files.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Covers open-compose shortcut, multiple-dialog stacking, the Tab-inserts-literal-tab trap, attachments via DOM.setFileInputFiles on the visible dialog's input, and stable selectors for To/Subject/Body/Send.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(run): pass globals to exec so comprehensions resolve free vars
exec(sys.stdin.read()) inside main() passes different dicts for globals
and locals. Python comprehensions and generator expressions compile to a
nested function whose free-variable lookups can only see the globals
dict, so code like
for it in items:
low = it['text'].lower()
hit = any(k in low for k in KEYS) # NameError: 'low'
fails at module level under exec. `low` is stored in the exec locals
dict which the generator's implicit function cannot see.
Passing globals() as the only extra arg makes exec use the same dict
for both globals and locals, and comprehensions resolve cleanly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(skill): eyeball click coords from screenshots; no getBoundingClientRect
The previous Clicking bullet said "screenshot -> look -> click(x,y)" but
did not rule out roundtripping through js("...getBoundingClientRect()")
to compute coords. Agents coming from Playwright / Selenium habits tend
to locate-first-click-second even when the screenshot already shows the
target, which is slower and more brittle (hidden inputs placed at
x=-9999, CSS-transformed elements, pseudo-elements) than just reading
the pixel off the image.
Rewrite the bullet to explicitly suppress that reflex and scope the
DOM-fallback to elements with no visible geometry. Also replace the
loose "compositor level" phrasing with the actual mechanism: Chrome's
browser-process hit-testing, which is why clicks pass through iframes
/ shadow DOM / cross-origin without extra work.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reuses the existing `_open_chrome_inspect()` helper so a bare
`browser-harness <<PY ... PY` invocation recovers from the three
cold-start failures that previously forced a manual restart_daemon()
or a separate `browser-harness --setup` run:
1. Stale daemon — previous run's Chrome was killed but the daemon
still listens on the unix socket. Probe with a real CDP call
(`Target.getTargets`) and require "result" in the reply. A
`{"meta":"session"}` probe would not work because the daemon
answers meta requests from a cached Python dict even when its
CDP WebSocket to Chrome is dead.
2. Cold Chrome — log tail says "DevToolsActivePort not found" or
"not live yet". `_open_chrome_inspect()` uses AppleScript
`activate` which launches Chrome if absent, then opens the
inspect page.
3. Missing Allow on chrome://inspect — log tail says
"WS handshake failed ... 403". Same recovery: open the inspect
page, print one stderr hint ("click Allow..."), retry spawn once.
Net: +37/-22 lines, one function touched, no new helpers.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Captures the durable shape of LinkedIn's received-invitations page:
- URL filter slugs (PEOPLE_WITH_MUTUAL_CONNECTION, PEOPLE_WITH_MUTUAL_SCHOOL)
and the chip counter as authoritative remaining-count.
- Accept/Ignore aria-label formats — they differ from each other, so you
cannot derive one from the other.
- The "follows you" trap: Accept renders as <a href=current-URL>, not
<button>; .click() follows href and no click path (MouseEvent, CDP
Input.dispatchMouseEvent) triggers the accept handler. Route these to
Ignore or skip.
- Pagination: list renders ~10 rows, replaced by "is now a connection"
acknowledgments after accepts; reload the URL to fetch the next slice,
scrolling does nothing.
- "Take care when connecting" safety modal appears intermittently.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After the previous commit added early raises for LOGIN_REQUIRED and
ERROR in scrape_video(), two doc entries still described the old
behavior ("will succeed but status is LOGIN_REQUIRED"). Updated both
the Gotchas section and the 'What Requires a Browser' list to reflect
that scrape_video() now raises ValueError on age-restricted videos.
Two issues raised by automated review on PR #137:
1. max_results limit could be exceeded
The `break` in youtube_search() only exits the inner loop over
itemSectionRenderer contents. If YouTube returns multiple sections,
the outer loop would continue appending results beyond max_results.
Fixed by replacing `break` with `return results` to exit both loops
immediately once the limit is reached.
2. Regex match not null-checked before .group(1)
scrape_video() called m.group(1) directly after re.search(), which
raises AttributeError if the pattern is not found (e.g. private video,
deleted video, region-blocked content, or YouTube HTML structure change).
Fixed by adding an explicit None check with a descriptive ValueError,
and an early playabilityStatus check that surfaces LOGIN_REQUIRED and
ERROR states with clear messages before attempting to parse videoDetails.
Adds domain-skills/youtube/scraping.md with four verified approaches
for extracting YouTube data without a browser or API key.
## What's included
**Approach 1 — oEmbed API (fastest)**
- Single HTTP call, ~0.3s per video
- Returns title, author, channel URL, thumbnail, embed HTML
- Bulk fetching via ThreadPoolExecutor with real timing data
- Verified on multiple video IDs
**Approach 2 — Watch page ytInitialPlayerResponse**
- Full video metadata: title, author, channel_id, duration, view_count,
publish_date, upload_date, category, like_count, keywords, is_live,
is_private, is_unlisted, available_countries (249 codes), embed_url
- Correct regex pattern (non-greedy with lookahead) to parse the JSON blob
- Real output values verified and included as comments
- Accurate gotchas: viewCount/lengthSeconds are strings not ints,
likeCount lives in microformat not videoDetails
**Approach 3 — Search results (no API key)**
- Parses ytInitialData from /results?search_query= (server-side rendered)
- Returns up to ~14-20 results with videoId, title, channel, duration,
views, published, description snippet, thumbnail URL
- Verified: 15 results returned for "python tutorial"
**Approach 4 — Channel metadata**
- Handles both @handle and channel ID (UC...) URL formats
- Extracts channel_id, title, description, subscriber count, avatar,
banner from pageHeaderViewModel + channelMetadataRenderer
- Verified on @RickAstleyYT: "4.48m subscribers"
**Utilities**
- thumbnail_urls(): all 5 sizes with availability notes (maxres may 404)
- extract_video_id(): handles watch, youtu.be, /shorts/, /embed/ formats
**What requires a browser**
- Clear list of what http_get cannot access: trending, playlists,
comments, caption text, age-restricted videos
**URL patterns reference table**
**Gotchas (all verified)**
- ytInitialPlayerResponse regex non-greedy requirement
- viewCount/lengthSeconds are string types, not int
- likeCount location (microformat, not videoDetails)
- oEmbed 404 on private/deleted videos
- Caption baseUrl returns empty in all tested conditions
(plain http_get, XHR, and fetch with cookies) — not a session issue
- Search result count varies (~14-20), never assume fixed count
- Subscriber count is a rounded string, not an integer
run_doctor() assigned `cur = _version() or "(unknown)"` and then fed
that placeholder into `_version_tuple()`. The parser ignores
non-leading-digit characters, so `"(unknown)"` parsed as `(0,)` and
any known `latest` compared as newer — falsely flagging an update.
run_update() short-circuited on `latest and not newer` without
checking `cur`. When cur was empty, `newer` was False via the
short-circuit in check_for_update, so it wrongly printed
`up to date ()` and returned 0 instead of attempting the update.
Keep `_version()` for the comparison and use the placeholder only
for display. In run_update, require `cur` before treating
`not newer` as up-to-date; otherwise proceed with the update.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Keeps tag-version verification and `uv build` on v* tag pushes but no
longer publishes to PyPI. Removes the OIDC id-token permission and the
`pypi` environment block that existed only for trusted publishing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Add --version, --doctor, --setup, --update[-y] commands in run.py;
logic lives in admin.py (install-mode detection, GitHub-releases cache
with 24h TTL, dirty-worktree guard, interactive Chrome-attach flow).
- Print a once-per-day startup banner telling agents to run
`browser-harness --update -y` when a newer release is available.
- Rename project to browser-harness in pyproject.toml so PyPI installs
(uv tool install browser-harness) work via the public package name.
- Add .github/workflows/release.yml: on v* tag push, verify the tag
matches pyproject.toml, uv build, and publish to PyPI via trusted
publishing.
- Wire helpers.http_get through fetch_use.fetch_sync(...).text when
BROWSER_USE_API_KEY is set; falls back to the original urllib path
otherwise, preserving the existing str return contract.
- Document the new commands and the agent's self-update duty in
SKILL.md and install.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When start_remote_daemon() fails because the cloud WebSocket is rejected
(e.g. data-center IP is blocked, cdpUrl expired, or remote browser was
stopped), the raised RuntimeError pointed users at the local "click
Allow in Chrome" flow, which is irrelevant on the cloud code path.
Branch on BU_CDP_WS at the handshake-failure catch site: when the daemon
was launched with a cloud WS URL, raise a cloud-specific message naming
the likely causes; otherwise keep the existing local-Chrome message.
Both paths preserve the underlying exception inline so stack-trace
context is retained.
Closes#108
The Fast start snippet used https://browser-use.com purely as an
illustrative "hello world". Pointing it at docs.browser-use.com
gives agents a more useful landing page on their first run.
Refs #102
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Captures the site shape for centilebrain.org 'Generate Estimates' flow:
- Six Shiny iframes (modality x sex) at centilebrain-app.shinyapps.io
- Selectors for email, file input, compute, download
- Wait markers and three non-obvious traps (iframe target_id staleness,
MUI switch checkboxes, coordinate-vs-JS button click after scroll)
- End-to-end example using upload_file(..., target_id=...) with
iframe_target()
- Output zip schema
Requires the iframe target_id support added to upload_file().
Saurav Panda