axolotl-ai-cloud--axolotl
wehub-resource-sync
78ec5d9290
ci-cd / build-axolotl-uv (<nil>, 130, 13.0.0, linux/amd64,linux/arm64, 3.12, 2.11.0) (push) Has been cancelled
ci-cd / build-axolotl-uv (<nil>, 130, 13.0.0, true, linux/amd64,linux/arm64, 3.12, 2.12.0) (push) Has been cancelled
ci-cd / build-axolotl-cloud-uv (<nil>, 130, 13.0.0, linux/amd64,linux/arm64, 3.12, 2.11.0) (push) Has been cancelled
ci-cd / build-axolotl-cloud-uv (<nil>, 130, 13.0.0, true, linux/amd64,linux/arm64, 3.12, 2.12.0) (push) Has been cancelled
ci-cd / build-axolotl-cloud-no-tmux-uv (<nil>, 130, 13.0.0, linux/amd64,linux/arm64, 3.12, 2.11.0) (push) Has been cancelled
ci-cd / build-axolotl-cloud-no-tmux-uv (<nil>, 130, 13.0.0, true, linux/amd64,linux/arm64, 3.12, 2.12.0) (push) Has been cancelled
ci-cd-base / build-base-uv (130, 13.0.0, , Dockerfile-uv-base, linux/amd64,linux/arm64, 3.12, 2.12.0, 9.0 10.0 10.3 12.0+PTX) (push) Has been cancelled
ci-cd-base / build-base-uv (130, 13.0.0, , Dockerfile-uv-base, linux/amd64,linux/arm64, 3.12, 2.12.1, 9.0 10.0 10.3 12.0+PTX) (push) Has been cancelled
ci-cd-base / build-base-uv (130, 13.0.0, , Dockerfile-uv-base, linux/amd64,linux/arm64, 3.12, 2.13.0, 9.0 10.0 10.3 12.0+PTX) (push) Has been cancelled
ci-cd-base / build-base-uv (132, 13.2.1, , Dockerfile-uv-base, linux/amd64,linux/arm64, 3.12, 2.13.0, 9.0 10.0 10.3 12.0+PTX, https://download.pytorch.org/whl/cu132) (push) Has been cancelled
ci-cd-base / build-base-uv (130, 13.0.0, , Dockerfile-uv-base, linux/amd64,linux/arm64, 3.12, 2.11.0, 9.0 10.0 10.3 12.0+PTX) (push) Has been cancelled
Tests / PyTest (3.12, 2.12.1) (push) Has been cancelled
Tests / PyTest (3.12, 2.13.0) (push) Has been cancelled
docker-e2e-tests / gate-skip-e2e (push) Has been cancelled
docker-e2e-tests / docker-e2e-tests-1st (<nil>, 130, 13.0.0, 1, 3.12, 2.12.1) (push) Has been cancelled
docker-e2e-tests / docker-e2e-tests (<nil>, 130, 13.0.0, 1, 3.12, 2.11.0) (push) Has been cancelled
docker-e2e-tests / docker-e2e-kernel-tests (<nil>, 130, 13.0.0, 1, 3.12, 2.11.0) (push) Has been cancelled
docker-e2e-tests / docker-e2e-kernel-tests (<nil>, 130, 13.0.0, 1, 3.12, 2.12.1) (push) Has been cancelled
docker-e2e-tests / docker-e2e-cleanup (<nil>, 130, 13.0.0, 1, 3.12, 2.12.1) (push) Has been cancelled
Publish Docs / build-deploy (push) Has been cancelled
Tests / PyTest from Source Dist (3.12, 2.11.0) (push) Has been cancelled
Tests / PyTest from Source Dist (3.12, 2.12.1) (push) Has been cancelled
Tests / PyTest from Source Dist (3.12, 2.13.0) (push) Has been cancelled
Tests / pre-commit (push) Has been cancelled
Tests / Prefetch S3 once to prime the CDN cache (push) Has been cancelled
Tests / PyTest (3.12, 2.11.0) (push) Has been cancelled
62 行
2.4 KiB
Plaintext
62 行
2.4 KiB
Plaintext
---
|
|
title: Telemetry
|
|
description: A description of the telemetry implementation in Axolotl.
|
|
---
|
|
|
|
# Telemetry in Axolotl
|
|
|
|
Axolotl implements anonymous telemetry to help maintainers understand how the library
|
|
is used and where users encounter issues. This data helps prioritize features, optimize
|
|
performance, and fix bugs.
|
|
|
|
## Data Collection
|
|
|
|
We collect:
|
|
|
|
- System info: OS, Python version, Axolotl version, PyTorch version, Transformers
|
|
version, etc.
|
|
- Hardware info: CPU count, memory, GPU count and models
|
|
- Runtime metrics: Training progress, memory usage, timing information
|
|
- Usage patterns: Models (from a whitelist) and configurations used
|
|
- Error tracking: Stack traces and error messages (sanitized to remove personal
|
|
information)
|
|
|
|
Personally identifiable information (PII) is not collected.
|
|
|
|
## Implementation
|
|
|
|
Telemetry is implemented using PostHog and consists of:
|
|
|
|
- `axolotl.telemetry.TelemetryManager`: A singleton class that initializes the
|
|
telemetry system and provides methods for tracking events.
|
|
- `axolotl.telemetry.errors.send_errors`: A decorator that captures exceptions and
|
|
sends sanitized stack traces.
|
|
- `axolotl.telemetry.runtime_metrics.RuntimeMetricsTracker`: A class that tracks
|
|
runtime metrics during training.
|
|
- `axolotl.telemetry.callbacks.TelemetryCallback`: A Trainer callback that sends
|
|
runtime metrics telemetry.
|
|
|
|
The telemetry system will block training startup for 10 seconds to ensure users are
|
|
aware of data collection, unless telemetry is explicitly enabled or disabled.
|
|
|
|
## Opt-Out Mechanism
|
|
|
|
Telemetry is **enabled by default** on an opt-out basis. To disable it, set
|
|
`AXOLOTL_DO_NOT_TRACK=1` or `DO_NOT_TRACK=1`.
|
|
|
|
A warning message will be logged on start to clearly inform users about telemetry.
|
|
We will remove this after some period.
|
|
|
|
To hide the warning message about telemetry that is displayed on train, etc. startup,
|
|
explicitly set: `AXOLOTL_DO_NOT_TRACK=0` (enable telemetry) or `AXOLOTL_DO_NOT_TRACK=1`
|
|
(explicitly disable telemetry).
|
|
|
|
## Privacy
|
|
|
|
- All path-like config information is automatically redacted from telemetry data
|
|
- Model information is only collected for whitelisted organizations
|
|
- See `axolotl/telemetry/whitelist.yaml` for the set of whitelisted organizations
|
|
- Each run generates a unique anonymous ID
|
|
- This allows us to link different telemetry events in a single same training run
|
|
- Telemetry is only sent from the main process to avoid duplicate events
|