{ "cells": [ { "cell_type": "markdown", "id": "f95d2373", "metadata": {}, "source": [ "# OpenCode as a txtai LLM\n", "\n", "[OpenCode](https://github.com/anomalyco/opencode) is an open source AI coding agent. It's rapidly growing in popularity as an open alternative to [Claude Code](https://code.claude.com/docs/en/overview). \n", "\n", "OpenCode shows the power of permissive open source. It's rapidly undergoing mass adoption and as of January 2026 sits at `81K+` GitHub ⭐'s. OpenCode supports running as a [local server](https://opencode.ai/docs/server/) via `opencode serve`.\n", "\n", "This notebook will explore how TxtAI integrates this as a LLM pipeline." ] }, { "cell_type": "markdown", "id": "a8719b92", "metadata": {}, "source": [ "# Install dependencies\n", "\n", "Install `txtai` and all dependencies." ] }, { "cell_type": "code", "execution_count": null, "id": "e00c85bc", "metadata": { "vscode": { "languageId": "shellscript" } }, "outputs": [], "source": [ "%%capture\n", "!pip install git+https://github.com/neuml/txtai#egg=txtai[api,pipeline-llm]\n", "\n", "# Install OpenCode if not already installed and run serve\n", "!curl -fsSL https://opencode.ai/install | bash\n", "!opencode serve &" ] }, { "cell_type": "markdown", "id": "73ffe60e", "metadata": {}, "source": [ "# Create a OpenCode LLM\n", "\n", "Now that everything is installed, let's test it out!" ] }, { "cell_type": "code", "execution_count": 6, "id": "0e35a304", "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "```python\n", "print(\"Hello, World!\")\n", "```" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from IPython.display import Markdown, display\n", "\n", "from txtai import LLM\n", "\n", "def generate(request):\n", " display(Markdown(llm(request)))\n", "\n", "# Connect to OpenCode server and use default LLM provider\n", "llm = LLM(\"opencode\")\n", "\n", "generate(\"Show a Python Hello World example\")" ] }, { "cell_type": "markdown", "id": "55dd0fad", "metadata": {}, "source": [ "OK, now let's try this for a more advanced use case." ] }, { "cell_type": "code", "execution_count": 8, "id": "b3080841", "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "```python\n", "from txtai import Embeddings, LLM\n", "\n", "# Create embeddings model\n", "embeddings = Embeddings(path=\"sentence-transformers/all-MiniLM-L6-v2\")\n", "\n", "# Create LLM\n", "llm = LLM(\"huggingface/Qwen/Qwen2.5-1.5B-Instruct\")\n", "\n", "# Sample documents\n", "documents = [\n", " \"Python is a programming language created by Guido van Rossum\",\n", " \"Machine learning is a subset of artificial intelligence\",\n", " \"Deep learning uses neural networks with multiple layers\"\n", "]\n", "\n", "# Build the index\n", "embeddings.index([(i, doc) for i, doc in enumerate(documents)])\n", "\n", "# RAG function\n", "def rag_query(question):\n", " # Search for relevant documents\n", " results = embeddings.search(question, 2)\n", " \n", " # Build context from retrieved documents\n", " context = \" \".join([documents[result[0]] for result in results])\n", " \n", " # Generate response using LLM with context\n", " prompt = f\"Context: {context}\\n\\nQuestion: {question}\\n\\nAnswer:\"\n", " return llm(prompt)\n", "\n", "# Example usage\n", "question = \"What is Python?\"\n", "answer = rag_query(question)\n", "print(f\"Question: {question}\")\n", "print(f\"Answer: {answer}\")\n", "```" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "generate(\"Show a simple and basic TxtAI RAG example\")" ] }, { "cell_type": "markdown", "id": "b44e0309", "metadata": {}, "source": [ "Interesting! It's not limited to Python though." ] }, { "cell_type": "code", "execution_count": 9, "id": "0a2ec1b4", "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "```assembly\n", "section .data\n", " hello db 'Hello, World!', 10 ; String with newline\n", " hello_len equ $ - hello ; Length of string\n", "\n", "section .text\n", " global _start\n", "\n", "_start:\n", " ; sys_write system call\n", " mov eax, 4 ; sys_write\n", " mov ebx, 1 ; stdout\n", " mov ecx, hello ; message\n", " mov edx, hello_len ; message length\n", " int 0x80 ; kernel interrupt\n", "\n", " ; sys_exit system call\n", " mov eax, 1 ; sys_exit\n", " mov ebx, 0 ; exit code 0\n", " int 0x80 ; kernel interrupt\n", "```\n", "\n", "Compile and run:\n", "```bash\n", "nasm -f elf32 hello.asm -o hello.o\n", "ld -m elf_i386 hello.o -o hello\n", "./hello\n", "```" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "generate(\"Show a x86 assembly hello world\")" ] }, { "cell_type": "code", "execution_count": 11, "id": "4e28d5c4", "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "```python\n", "a = None\n", "if a is not None:\n", " a.startswith(\"hello\")\n", "else:\n", " print(\"a is None, cannot call startswith\")\n", "```\n", "\n", "**What's wrong:** The variable `a` is `None`, which is a `NoneType` object. The `startswith()` method only exists for strings, not for `NoneType`. You need to check if `a` is not `None` before calling string methods on it." ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "generate(\"\"\"\n", "Fix the following error in the code below and explain what's wrong.\n", "\n", "Error:\n", "AttributeError: 'NoneType' object has no attribute 'startswith'\n", "\n", "Code:\n", "a = None\n", "a.startswith(\"hello\")\n", "\"\"\")" ] }, { "cell_type": "markdown", "id": "156122f0", "metadata": {}, "source": [ "# OpenAI-compatible Endpoint\n", "\n", "Now that OpenCode is integrated into the TxtAI ecosystem, it opens a lot of different opportunities. Let's say we want to host a local OpenAI compatible endpoint, no problem!\n", "\n", "Write the following file.\n", "\n", "## config.yml\n", "\n", "```yaml\n", "# Enable OpenAI compat endpoint\n", "openai: True\n", "\n", "llm:\n", " path: opencode\n", "```\n", "\n", "and start the following process.\n", "\n", "```\n", "CONFIG=config.yml uvicorn \"txtai.api:app\"\n", "```" ] }, { "cell_type": "code", "execution_count": 16, "id": "33057f59", "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "```python\n", "def factorial(n):\n", " if n < 0:\n", " raise ValueError(\"Factorial is not defined for negative numbers\")\n", " if n == 0 or n == 1:\n", " return 1\n", " result = 1\n", " for i in range(2, n + 1):\n", " result *= i\n", " return result\n", "\n", "# Example usage:\n", "# print(factorial(5)) # Output: 120\n", "```" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from IPython.display import Markdown, display\n", "\n", "from openai import OpenAI\n", "\n", "client = OpenAI(\n", " base_url=\"http://localhost:8000/v1\",\n", " api_key=\"api-key\",\n", ")\n", "\n", "response = client.chat.completions.create(\n", " messages=[{\n", " \"role\": \"user\",\n", " \"content\": \"Show a Python factorial function\",\n", " }],\n", " model=\"opencode\",\n", ")\n", "\n", "display(Markdown((response.choices[0].message.content)))" ] }, { "cell_type": "markdown", "id": "58a9169b", "metadata": {}, "source": [ "Just like before we get an answer! This time via the OpenAI client. Fun times." ] }, { "cell_type": "markdown", "id": "47f9dbbd", "metadata": {}, "source": [ "# Wrapping up\n", "\n", "This notebook showed how to connect TxtAI and OpenCode. This will lead to some interesting integrations. One such example is [ncoder](https://github.com/neuml/ncoder), an AI coding agent that integrates with Jupyter Notebooks. Stay tuned for more!" ] } ], "metadata": { "kernelspec": { "display_name": "local", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.19" } }, "nbformat": 4, "nbformat_minor": 5 }