- daemon.py:1, SKILL.md:151 — docstring/diagram said named pipe on Windows;
the implementation is TCP loopback. Updated.
- daemon.py:209 — if ipc.serve() crashes (e.g. bind failure), the prior
shutdown path could miss it and leave the daemon waiting forever on
d.stop without a listening endpoint. Now race serve_task and stop.wait()
via asyncio.wait(FIRST_COMPLETED): if serve finishes first it must have
raised, so await it to surface the exception. Cleanup cancels both tasks
unconditionally.
- ipc.py:45 — BU_NAME flowed straight into f-strings building filesystem
paths, allowing path traversal outside tempdir. Validate via
^[A-Za-z0-9_-]{1,64}$ in a single _check() helper that all path
builders (log_path/pid_path/port_path/_sock_path) call. Bad names raise
ValueError early with a clear message.
The harness was Linux/macOS-only because daemon IPC hardcoded AF_UNIX sockets
at /tmp/bu-*.sock paths and asyncio.start_unix_server, all of which are
unavailable or invalid on Windows. Worse, uv-managed Python on Windows
(python-build-standalone) ships without socket.AF_UNIX entirely (#124).
New ipc.py centralizes the platform fork:
- POSIX: AF_UNIX socket at <tempdir>/bu-<NAME>.sock (chmod 0600), unchanged
semantics from the prior /tmp-hardcoded path.
- Windows: TCP loopback on 127.0.0.1:<ephemeral>, with the chosen port
written to <tempdir>/bu-<NAME>.port so clients can find the daemon.
Uses asyncio.start_server (stdlib, no obscure APIs, no third-party deps).
Path discipline: log/pid/port files all sit under tempfile.gettempdir() so
they land in /tmp on Linux, $TMPDIR on macOS, %TEMP% on Windows. helpers.py
screenshot() default also moves from /tmp/shot.png to tempfile.gettempdir().
subprocess detach uses start_new_session=True on POSIX and
DETACHED_PROCESS|CREATE_NEW_PROCESS_GROUP on Windows via ipc.spawn_kwargs().
run.py reconfigures stdout to UTF-8 on Windows so print(page_info()) doesn't
UnicodeEncodeError on the 🟢 marker that helpers prepend to tab titles
(#124 item 4). cp1252 (PowerShell default) can't encode it.
Verified end-to-end on Windows 11 with Chrome remote debugging:
- daemon spawns, allocates port, writes .port file
- goto + page_info + screenshot round-trip through TCP loopback
- restart_daemon cleans up .port and .pid
POSIX path is logically equivalent to the prior code (same AF_UNIX call,
same socket-file semantics, same chmod 0600), routed through ipc.py.
Closes#124 items 1, 2, 4. Item 3 (Chrome 147 user-data-dir) is a separate
concern not addressed here.
Bot review caught two issues:
1. capture_screenshot's max_dim path imports PIL but Pillow wasn't in
pyproject.toml dependencies, so a clean install would ImportError as
soon as the option was used.
2. NamedTemporaryFile + writing to f.name while the handle is open is
not portable on Windows (file lock). Switched the screenshot tests to
tempfile.TemporaryDirectory and folded the three near-identical bodies
into a small _run() helper.
Long agent sessions on 2× displays bust the 2000px-per-side limit some
image-aware LLMs enforce — a 2296×1143 CSS viewport produces a 4592×2286
PNG. Passing max_dim=1800 downscales the file before save (only when the
image actually exceeds max_dim), keeping callers that don't pass it
unchanged.
Covers URL-based search (dates, destination, travellers), traveller widget
JS interaction, child age dropdowns, and price filter usage. Documents that
the date picker is unreliable with coordinate clicks and should be bypassed
via URL parameters.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Three places still described paywalled post body_html as null/None:
- TL;DR limitations list
- Approach 2 section header
- substack_get_post docstring
All updated to correctly state that body_html is a truncated HTML preview
(not null) for paywalled posts, consistent with the Gotchas section and
empirical verification.
Adds domain-skills/substack/scraping.md with four field-tested approaches
for extracting data from any Substack publication without authentication
or a browser session. All approaches verified live on 2026-04-27.
Approach 1 — /api/v1/posts: paginated post list with title, slug, audience,
wordcount, reactions, and post_id. Supports offset pagination.
Approach 2 — /api/v1/posts/{slug}: full post content. Returns complete
body_html (~40KB) for free posts; truncated HTML preview for paywalled posts.
Use audience == "everyone" as the reliable signal for full content.
Approach 3 — /api/v1/post/{id}/comments: comment list with author, body,
date, and reaction counts. Uses integer post_id (not slug).
Approach 4 — /feed: lightweight RSS metadata (title/link/pubDate/description)
without JSON parsing overhead.
Covers both URL formats (native subdomain and custom domain), pagination,
paywalled post handling, and gotchas: reactions is an emoji-keyed dict not
an integer, comments endpoint uses post_id not slug, body_html is a truncated
preview (not null) for paid posts, no unauthenticated cross-publication search.
Co-Authored-By: Tianye Song <songtianye1997@gmail.com>
BU_CDP_URL takes a Chrome DevTools HTTP endpoint (e.g.
http://127.0.0.1:9333) and resolves it to the WS URL via /json/version,
mirroring how start_remote_daemon already handles cloud browsers.
The motivating use case is running a dedicated automation Chrome on a
non-default --user-data-dir to avoid both the Chrome 136 default-profile
lockdown and the Chrome 144+ "Allow remote debugging" per-connection
consent dialog. Pointing BU_CDP_URL at that instance lets the harness
attach without prompting the user.
Falls back to the existing default-profile DevToolsActivePort discovery
when neither BU_CDP_WS nor BU_CDP_URL is set, so existing setups are
unaffected.
Running `browser-harness -c` without a code argument crashed with an
unhandled IndexError at `exec(args[1])`. Added a length check to
produce a proper usage message instead.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a domain skill for enumerating Loom library folders
(loom.com/looms/videos/<slug>-<id>) — the private workspace variant
that requires an authenticated session.
yt-dlp's existing LoomFolderIE covers public-shared folders
(loom.com/share/folder/<id>) but the underlying /v1/folders/<id>
endpoint returns Forbidden for library IDs even with cookies.
That makes browser-harness with the user's open Chrome tab the
only practical programmatic route for private workspace content.
Field-tested on a 78-video folder; documents the data-videoid
selector, the scrollIntoView mechanic that beats scrollTop's
silent cap, dead-end endpoints, and the pipe-to-yt-dlp setup.
current_tab() and list_tabs() both return dicts shaped {"targetId": ...,
"url": ..., "title": ...}, but switch_tab() only accepted the bare string.
This made the natural pattern
original = current_tab()
new_tab(...)
switch_tab(original)
raise InvalidParameters from the CDP layer ("string value expected at
position 17"). Callers had to remember to pull ["targetId"] out manually,
which contradicts the harness's "obvious shapes work" ergonomic.
Make switch_tab tolerant of either shape: pull targetId from the dict if
one is passed, otherwise treat the input as the id string.
* fix(js): don't double-wrap IIFEs that contain return
The substring check `"return " in expression` incorrectly matched
expressions that already contained an IIFE with an internal return,
causing double-wrapping and a silent None result. Guard with
`not expression.strip().startswith("(")` so pre-wrapped expressions
pass through unchanged.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(cli): add --reload flag to restart the daemon
Kills the running daemon so the next call picks up code changes,
replacing the manual pkill workflow after editing helpers.py.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
The substring check `"return " in expression` incorrectly matched
expressions that already contained an IIFE with an internal return,
causing double-wrapping and a silent None result. Guard with
`not expression.strip().startswith("(")` so pre-wrapped expressions
pass through unchanged.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
When falling through to the browser path for watch-page DOM (instead of
the http_get + ytInitialPlayerResponse blob), wait_for_load() is not
enough. The load event fires before YouTube's Polymer components
hydrate — h1.ytd-watch-metadata yt-formatted-string,
ytd-video-owner-renderer #channel-name a, and ytd-watch-info-text all
return null for ~2s after load. A wait(3) after wait_for_load() is
required before querying any watch-page selector.
Field-tested 2026-04-24 on Brave; same behavior observed on
ungoogled-chromium. The HTTP path remains the recommended approach for
metadata; this note exists for the cases that genuinely need the
rendered DOM (live UI state, etc.).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Markdown-style domain skill for Polymarket scraping via Gamma API
(api-first per repo doctrine) with DOM leaf-div-disambiguation fallback
for CSS-module SPAs. Covers market outcomes, metadata, and comments.
Live-tested against gamma-api.polymarket.com and a live event page:
- 9 outcomes extracted (e.g. April 7: YES 99.95 / NO 0.05, vol $45.7M)
- Metadata: title, end_date, total_volume, category, market_count
- 38 comments fetched (40 raw, 2 deleted skipped)
Gotcha documented: Gamma API comment envelopes for deleted comments
preserve id/createdAt/profile/media/parentCommentID but drop the body
field entirely — naive dict access throws KeyError. Guard with
'if "body" not in c: continue'.
DOM fallback pattern documented (not primary path): Polymarket has
zero data-testid attributes and CSS-module-hashed classes. Leaf-div
disambiguation (children.length === 0 + nearest-common-ancestor
grouping) is the only robust approach.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- manus/tasks.md: ProseMirror composer can mount late; wrap the
getBoundingClientRect lookup in a bounded retry + assertion so the
helper doesn't null-deref and block submission.
- perplexity/computer.md: id extraction must parse the URL pathname
before slicing — raw-href slicing corrupts when the URL carries
?view=thread or a hash fragment.
- perplexity/computer.md: all_todo_done() returned false on hydrate
while the doc claimed None; unify on None so "can't tell" states
(closed panel, empty panel) are distinguishable from "done=false".
When BH_DEBUG_CLICKS=1 (or --debug-clicks CLI flag), click_at_xy
captures a screenshot before each click and draws a red crosshair
at the exact click location, scaled by window.devicePixelRatio so
the marker aligns correctly on HiDPI/Retina displays.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(js): auto-wrap top-level return expressions in an IIFE
Agents naturally write js() calls with top-level `return`, which is a
syntax error in Runtime.evaluate. js() now detects `return ` and wraps
the expression in an IIFE so both styles work without callers needing
to know the CDP constraint.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor(run): replace heredoc stdin with -c flag
Drops the <<'PY' heredoc pattern entirely. Agents now pass code via
`browser-harness -c "..."`, which is simpler to construct programmatically
and avoids heredoc quoting pitfalls.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Agents naturally write js() calls with top-level `return`, which is a
syntax error in Runtime.evaluate. js() now detects `return ` and wraps
the expression in an IIFE so both styles work without callers needing
to know the CDP constraint.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
SKILL.md now covers day-to-day usage only. Maintenance commands
(--doctor, --setup, --update) and the architecture section move to
install.md where setup and break-fix content belongs.
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
manus/tasks.md
- Correct the Connect RPC transport description: the JS client wraps the
JSON payload in a Uint8Array before fetch, so a naive fetch-hook logs
it as <<bytes:N>> — the wire content matches content-type (JSON both
ways), not a separate binary codec.
- Replace the raw click(750, 505) example with a computed rect from
div.ProseMirror[contenteditable="true"] (SKILL.md: no pixel coords).
- Drop the specific-task prompt example ('reply with just the word PONG'
→ 'Reply with PONG') for a generic description of the auto-summary.
- Remove a real taskId from the URL-pattern docstring.
perplexity/computer.md
- Scope Todo-panel queries to the Radix popover. The Todo button has
data-state / aria-expanded / aria-controls; the open panel is mounted
with id === button.aria-controls. A global 'svg use' query across the
whole document picks up tool-invocation icons, sidebar icons, and top-
bar icons — scope-less completion detection is unreliable.
- Rewrite all_todo_done() to return None when the panel is closed or
hydrating and True only when every row's icon is #pplx-icon-check.
- Remove a real slug and taskId from the URL-pattern docstring.
- goto() → goto_url()
- click() → click_at_xy()
- screenshot() → capture_screenshot()
These three shared exact names with Playwright but different argument
shapes, causing agent confusion. Updated helpers.py and all markdown
skill files.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Covers open-compose shortcut, multiple-dialog stacking, the Tab-inserts-literal-tab trap, attachments via DOM.setFileInputFiles on the visible dialog's input, and stable selectors for To/Subject/Body/Send.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two small fixes in daemon.py for macOS users running Brave (or any
Chromium variant launched with --remote-debugging-port):
1. PROFILES list was missing the macOS path for Brave
(~/Library/Application Support/BraveSoftware/Brave-Browser).
The Linux Flatpak path was the only Brave entry, so macOS Brave
users got "DevToolsActivePort not found" even after enabling
remote debugging.
2. When Chromium is launched with an explicit --remote-debugging-port
flag, the DevToolsActivePort file is not always written to the
profile dir. CDP is fully live, but get_ws_url() can't find it.
Added a final fallback that probes 127.0.0.1:9222 and :9223 via
/json/version and uses the returned webSocketDebuggerUrl. The
loop is gated to the standard debugging ports and only runs after
the existing profile-dir scan fails, so it doesn't interfere with
the normal sticky-checkbox flow.
Both changes are additive — no existing behavior is altered.
Alezander9