run-llama--llama_index
wehub-resource-sync
a0c8464e58
Build Package / build (ubuntu-latest) (push) Failing after 1s
CodeQL / Analyze (python) (push) Failing after 1s
Core Typecheck / core-typecheck (push) Failing after 1s
Linting / lint (push) Failing after 1s
llama-dev tests / test-llama-dev (push) Failing after 1s
Publish Sub-Package to PyPI if Needed / publish_subpackage_if_needed (push) Has been skipped
Sync Docs to Developer Hub / sync-docs (push) Failing after 0s
Build Package / build (windows-latest) (push) Has been cancelled
304 行
8.6 KiB
Plaintext
304 行
8.6 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "4ec7cd6e",
|
|
"metadata": {},
|
|
"source": [
|
|
"<a href=\"https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/examples/llm/predibase.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "bf9f19f3",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Predibase\n",
|
|
"\n",
|
|
"This notebook shows how you can use Predibase-hosted LLM's within Llamaindex. You can add [Predibase](https://predibase.com) to your existing Llamaindex worklow to: \n",
|
|
"1. Deploy and query pre-trained or custom open source LLM’s without the hassle\n",
|
|
"2. Operationalize an end-to-end Retrieval Augmented Generation (RAG) system\n",
|
|
"3. Fine-tune your own LLM in just a few lines of code\n",
|
|
"\n",
|
|
"## Getting Started\n",
|
|
"1. Sign up for a free Predibase account [here](https://predibase.com/free-trial)\n",
|
|
"2. Create an Account\n",
|
|
"3. Go to Settings > My profile and Generate a new API Token."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "72d6eb5b",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"%pip install llama-index-llms-predibase"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "79a726c5",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!pip install llama-index --quiet\n",
|
|
"!pip install predibase --quiet\n",
|
|
"!pip install sentence-transformers --quiet"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "1c2b0d5d",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import os\n",
|
|
"\n",
|
|
"os.environ[\"PREDIBASE_API_TOKEN\"] = \"{PREDIBASE_API_TOKEN}\"\n",
|
|
"from llama_index.llms.predibase import PredibaseLLM"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "9a602a2a",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Flow 1: Query Predibase LLM directly"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "4baffaa2",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Predibase-hosted fine-tuned adapter example\n",
|
|
"llm = PredibaseLLM(\n",
|
|
" model_name=\"mistral-7b\",\n",
|
|
" predibase_sdk_version=None, # optional parameter (defaults to the latest Predibase SDK version if omitted)\n",
|
|
" adapter_id=\"e2e_nlg\", # adapter_id is optional\n",
|
|
" adapter_version=1, # optional parameter (applies to Predibase only)\n",
|
|
" api_token=None, # optional parameter for accessing services hosting adapters (e.g., HuggingFace)\n",
|
|
" max_new_tokens=512,\n",
|
|
" temperature=0.3,\n",
|
|
")\n",
|
|
"# The `model_name` parameter is the Predibase \"serverless\" base_model ID\n",
|
|
"# (see https://docs.predibase.com/user-guide/inference/models for the catalog).\n",
|
|
"# You can also optionally specify a fine-tuned adapter that's hosted on Predibase or HuggingFace\n",
|
|
"# In the case of Predibase-hosted adapters, you must also specify the adapter_version"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "69713553",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# HuggingFace-hosted fine-tuned adapter example\n",
|
|
"llm = PredibaseLLM(\n",
|
|
" model_name=\"mistral-7b\",\n",
|
|
" predibase_sdk_version=None, # optional parameter (defaults to the latest Predibase SDK version if omitted)\n",
|
|
" adapter_id=\"predibase/e2e_nlg\", # adapter_id is optional\n",
|
|
" api_token=os.environ.get(\n",
|
|
" \"HUGGING_FACE_HUB_TOKEN\"\n",
|
|
" ), # optional parameter for accessing services hosting adapters (e.g., HuggingFace)\n",
|
|
" max_new_tokens=512,\n",
|
|
" temperature=0.3,\n",
|
|
")\n",
|
|
"# The `model_name` parameter is the Predibase \"serverless\" base_model ID\n",
|
|
"# (see https://docs.predibase.com/user-guide/inference/models for the catalog).\n",
|
|
"# You can also optionally specify a fine-tuned adapter that's hosted on Predibase or HuggingFace\n",
|
|
"# In the case of Predibase-hosted adapters, you can also specify the adapter_version (assumed latest if omitted)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "e7039a65",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"result = llm.complete(\"Can you recommend me a nice dry white wine?\")\n",
|
|
"print(result)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "1112e828",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Flow 2: Retrieval Augmented Generation (RAG) with Predibase LLM"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "cacff36a",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from llama_index.core import VectorStoreIndex, SimpleDirectoryReader\n",
|
|
"from llama_index.core.embeddings import resolve_embed_model\n",
|
|
"from llama_index.core.node_parser import SentenceSplitter"
|
|
]
|
|
},
|
|
{
|
|
"attachments": {},
|
|
"cell_type": "markdown",
|
|
"id": "c8f6fef1",
|
|
"metadata": {},
|
|
"source": [
|
|
"#### Download Data"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "65930e7e",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!mkdir -p 'data/paul_graham/'\n",
|
|
"!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "1edd41d1",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Load Documents"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "c5941151",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"documents = SimpleDirectoryReader(\"./data/paul_graham/\").load_data()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "7df4407f",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Configure Predibase LLM"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "3f67e975-3cb5-4ddc-98e8-eae7892315ca",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Predibase-hosted fine-tuned adapter\n",
|
|
"llm = PredibaseLLM(\n",
|
|
" model_name=\"mistral-7b\",\n",
|
|
" predibase_sdk_version=None, # optional parameter (defaults to the latest Predibase SDK version if omitted)\n",
|
|
" adapter_id=\"e2e_nlg\", # adapter_id is optional\n",
|
|
" api_token=None, # optional parameter for accessing services hosting adapters (e.g., HuggingFace)\n",
|
|
" temperature=0.3,\n",
|
|
" context_window=1024,\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "4a44defc",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# HuggingFace-hosted fine-tuned adapter\n",
|
|
"llm = PredibaseLLM(\n",
|
|
" model_name=\"mistral-7b\",\n",
|
|
" predibase_sdk_version=None, # optional parameter (defaults to the latest Predibase SDK version if omitted)\n",
|
|
" adapter_id=\"predibase/e2e_nlg\", # adapter_id is optional\n",
|
|
" api_token=os.environ.get(\n",
|
|
" \"HUGGING_FACE_HUB_TOKEN\"\n",
|
|
" ), # optional parameter for accessing services hosting adapters (e.g., HuggingFace)\n",
|
|
" temperature=0.3,\n",
|
|
" context_window=1024,\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "b3d527b7-5110-4d9c-97df-3926d3db0772",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"embed_model = resolve_embed_model(\"local:BAAI/bge-small-en-v1.5\")\n",
|
|
"splitter = SentenceSplitter(chunk_size=1024)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "7a131a8e",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Setup and Query Index"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "c9b10269",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"index = VectorStoreIndex.from_documents(\n",
|
|
" documents, transformations=[splitter], embed_model=embed_model\n",
|
|
")\n",
|
|
"query_engine = index.as_query_engine(llm=llm)\n",
|
|
"response = query_engine.query(\"What did the author do growing up?\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "ac73eb65",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"print(response)"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3"
|
|
},
|
|
"vscode": {
|
|
"interpreter": {
|
|
"hash": "5ae9fa2777630f93d325d67fd0c37f7375ed1afcb20dd85f425eb8692a47ff3f"
|
|
}
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|