This PR completes the telemetry system by implementing the transport
layer for `ClearcutSender`. It enables actual HTTP communication with
the Clearcut backend, handling event batching, rate limiting, and
reliable delivery, including robust shutdown handling.
**Key Changes:**
* **HTTP Transport**: Implemented `fetch`-based transport sending `POST`
requests to the Clearcut HTTP server.
* **Event Batching**: Events are now buffered and flushed periodically
(default: 15 minutes) or on shutdown.
* **Reliability & Rate Limiting**:
* **Server-Side Backoff**: Respects `next_request_wait_millis` from
server responses to handle rate limiting dynamically.
* **Transient Error Retries**: Failed requests (5xx, 429) result in
events being requeued for the next flush.
* **Request Timeouts**: Enforced 30s timeout on requests to prevent
hanging processes.
* **Session Rotation**: Automatically rotates session IDs every 24
hours.
* **Safety & Stability**:
* **Buffer Overflow Protection**: Caps the buffer at 1000 events to
prevent memory leaks, dropping oldest events if necessary.
* **Optimistic Removal**: Prevents race conditions and duplicate events
during shutdown by optimistically removing events from the buffer before
sending.
* **Testing Improvements**:
* **E2E Robustness**: Updated E2E tests to use a mock web server instead
of relying on the logger to log specific lines.
**Implementation Roadmap:**
These changes finalize the planned telemetry architecture:
1. **CLI & Opt-out Mechanism
([Merged](https://github.com/ChromeDevTools/chrome-devtools-mcp/pull/757))**
2. **Logger Scaffolding & Integration
([Merged](https://github.com/ChromeDevTools/chrome-devtools-mcp/pull/758))**
3. **Persistence Layer
([Merged](https://github.com/ChromeDevTools/chrome-devtools-mcp/pull/766))**
4. **Watchdog Process Architecture
([Merged](https://github.com/ChromeDevTools/chrome-devtools-mcp/pull/769))**
5. **Transport, Batching & Retries (This PR):**
* Finalized `ClearcutSender` with HTTP transport, batching, and
server-directed backoff strategies.
---------
Co-authored-by: Alex Rudenko <alexrudenko@chromium.org>
This PR implements the watchdog process architecture for the telemetry
system. It moves the `ClearcutSender` execution to a dedicated child
process, ensuring that events—especially shutdown events—are reliably
transmitted even if the main server process terminates abruptly.
Added an e2e test that runs the server, checks the log file and confirms
the telemetry logs exist and that the watchdog process is correctly
killed after sending the shutdown event once the main process is killed.
**Implementation Roadmap:**
This is the fourth in a series of PRs designed to implement the
telemetry system:
1. **CLI & Opt-out Mechanism
([Merged](https://github.com/ChromeDevTools/chrome-devtools-mcp/pull/757)):**
* Added `--usage-statistics` flag and transparency logging.
2. **Logger Scaffolding & Integration
([Merged](https://github.com/ChromeDevTools/chrome-devtools-mcp/pull/758)):**
* **`ClearcutLogger`**: Implemented the main logging entry point.
* **One-way Data Flow**: Integrated `logToolInvocation` and
`logServerStart` hooks into `main.ts` to capture events.
* **`ClearcutSender`**: Introduced a transport abstraction.
* **Type Definitions**: Added TypeScript definitions for the telemetry
Protocol Buffer messages.
3. **Persistence Layer
([Merged](https://github.com/ChromeDevTools/chrome-devtools-mcp/pull/766)):**
* **`FilePersistence`**: Implemented a local file-based state manager to
persist the `lastActive` timestamp.
* **Daily Active Logic**: Integrated persistence into `ClearcutLogger`
to automatically detect and log `daily_active` events (with
`days_since_last_active` calculation) via `logDailyActiveIfNeeded`.
4. **Watchdog Process Architecture (This PR):**
* **`WatchdogClient`**: Added a client-side wrapper to spawn and
communicate with the watchdog process via `stdin`.
* **`watchdog/main.ts`**: Created the entry point for the watchdog
process. It listens for IPC messages and uses `ClearcutSender` to
transmit events.
* **Reliable Shutdown**: The watchdog monitors the parent process and
guarantees a `shutdown` event is sent when the parent exits or crashes
(detecting `stdin` closure).
* **Refactoring**: Moved `ClearcutSender` to the `watchdog` directory
and updated `ClearcutLogger` to delegate event sending to the
`WatchdogClient`.
5. **Transport, Batching & Retries (Next):**
* Finalize `ClearcutSender` with actual HTTP transport logic, including
event batching and exponential backoff retries.
Alex Rudenko