light-heart-labs--ods
wehub-resource-sync
52 行
2.0 KiB
Plaintext
52 行
2.0 KiB
Plaintext
# ODS: Local AI Infrastructure
|
|
|
|
ODS is a turnkey local AI stack designed to bring powerful AI capabilities to your own hardware. It provides a complete solution for running large language models, voice processing, and workflow automation entirely on your own infrastructure.
|
|
|
|
## Core Components
|
|
|
|
### llama-server - High Performance Inference
|
|
llama-server (from llama.cpp) serves as the backbone of ODS, providing OpenAI-compatible API endpoints for language model inference. It supports GGUF models including Qwen 3, Llama 3, and Mistral, automatically selecting the best model for your hardware tier.
|
|
|
|
### Open WebUI - Chat Interface
|
|
A beautiful, responsive chat interface that works with any OpenAI-compatible backend. Features include conversation history, model selection, and user management.
|
|
|
|
### Voice Processing
|
|
- **Whisper** for speech-to-text transcription with high accuracy
|
|
- **Piper** for natural-sounding text-to-speech synthesis
|
|
- Combined, these enable fully local voice assistants
|
|
|
|
### Workflow Automation
|
|
n8n integration provides visual workflow automation. Pre-built workflows include:
|
|
- Document Q&A (RAG pipeline)
|
|
- Voice transcription
|
|
- Code assistance
|
|
- Scheduled summarization
|
|
|
|
### Vector Database
|
|
Qdrant provides efficient vector storage and similarity search for RAG (Retrieval Augmented Generation) applications.
|
|
|
|
## Hardware Tiers
|
|
|
|
ODS automatically detects your GPU and configures appropriate models:
|
|
|
|
1. **Minimal** (<20GB VRAM): 7B quantized models
|
|
2. **Entry** (20-27GB VRAM): 14B AWQ models
|
|
3. **Prosumer** (28-47GB VRAM): 32B AWQ models
|
|
4. **Pro** (48GB+ VRAM): 70B+ models
|
|
|
|
## Installation
|
|
|
|
```bash
|
|
git clone https://github.com/Light-Heart-Labs/ODS.git
|
|
cd ODS/ods
|
|
./install.sh
|
|
```
|
|
|
|
The installer handles everything: Docker setup, GPU configuration, model selection, and service startup.
|
|
|
|
## Philosophy
|
|
|
|
ODS embodies the principle that AI should be accessible, private, and owned by users. Your data stays on your hardware. Your models run locally. Your AI, your rules.
|
|
|
|
Built by The Collective — making local AI practical for everyone.
|