llmware-ai--llmware
wehub-resource-sync
273 行
11 KiB
Plaintext
273 行
11 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "StkY5oHGU-iN"
|
|
},
|
|
"source": [
|
|
"# LLMWare Model Exploration\n",
|
|
"\n",
|
|
"## This is the 'entrypoint' example that provides a general introduction of llmware models.\n",
|
|
"\n",
|
|
"This notebook provides an introduction to LLMWare Agentic AI models and demonstrates their usage."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"collapsed": true,
|
|
"id": "KyaEnPzOVTJe"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# install dependencies\n",
|
|
"!pip3 install llmware"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "mcOxXgs1XTjD"
|
|
},
|
|
"source": [
|
|
"If you have any dependency install issues, please review the README, docs link, or raise an Issue.\n",
|
|
"\n",
|
|
"Usually, if there is a missing dependency, the code will give the warning - and a clear direction like `pip install transformers'` required for this example, etc.\n",
|
|
"\n",
|
|
"As an alternative to pip install ... if you prefer, you can also clone the repo from github which provides a benefit of having access to 100+ examples.\n",
|
|
"\n",
|
|
"To clone the repo:\n",
|
|
"```\n",
|
|
"git clone \"https://www.github.com/llmware-ai/llmware.git\"\n",
|
|
"sh \"welcome_to_llmware.sh\"\n",
|
|
"```\n",
|
|
"\n",
|
|
"The second script `\"welcome_to_llmware.sh\"` will install all of the dependencies.\n",
|
|
"\n",
|
|
"If using Windows, then use the `\"welcome_to_llmware_windows.sh\"` script."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "n4aKjcEiVjYE"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Import Library\n",
|
|
"from llmware.models import ModelCatalog"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "ePtRGBIlZEkP"
|
|
},
|
|
"source": [
|
|
"## GETTING STARTED WITH AGENTIC AI\n",
|
|
"All LLMWare models are accessible through the ModelCatalog generally consisting of two steps to access any model\n",
|
|
"\n",
|
|
"- Step 1 - load the model - pulls from global repo the first time, and then automatically caches locally\n",
|
|
"- Step 2 - use the model with inference or function call"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"collapsed": true,
|
|
"id": "D0xL5WOgVlGX"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# 'Standard' Models use 'inference' and take a general text input and provide a general text output\n",
|
|
"\n",
|
|
"model = ModelCatalog().load_model(\"bling-answer-tool\")\n",
|
|
"response = model.inference(\"My son is 21 years old.\\nHow old is my son?\")\n",
|
|
"\n",
|
|
"print(\"\\nresponse: \", response)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"collapsed": true,
|
|
"id": "1AkSZ3Z_VqWt"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Optional parameters can improve results\n",
|
|
"model = ModelCatalog().load_model(\"bling-phi-3-gguf\", temperature=0.0,sample=False, max_output=200)\n",
|
|
"\n",
|
|
"# all LLMWare models have been fine-tuned to assume that the input will include a text passage, and that the\n",
|
|
"# model's main job is to 'read' the passage, and then 'answer' a question based on that information\n",
|
|
"\n",
|
|
"text_passage = \"The company's stock price increased by $3 after reporting positive earnings.\"\n",
|
|
"prompt = \"What was the increase in the stock price?\"\n",
|
|
"\n",
|
|
"response = model.inference(prompt,add_context=text_passage)\n",
|
|
"\n",
|
|
"print(\"\\nresponse: \", response)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "OuNEktB-aPVw"
|
|
},
|
|
"source": [
|
|
"## Models we have and support\n",
|
|
"Inference models can also be integrated into Prompts - which provide advanced handling for integrating with knowledge retrieval, managing source information, and providing fact-checking\n",
|
|
"\n",
|
|
"Discovering other models is easy -> to invoke a model, simply use the `'model_name'` and pass in `.load_model()`.\n",
|
|
"\n",
|
|
"***note***: *model_names starting with `'bling'`, `'dragon'`, and `'slim'` are llmware models.*\n",
|
|
"- we do **include other popular models** such as `phi-3`, `qwen-2`, `yi`, `llama-3`, `mistral`\n",
|
|
"- it is easy to extend the model catalog to **include other 3rd party models**, including `ollama` and `lm studio`.\n",
|
|
"- we do **support** `open ai`, `anthropic`, `cohere` and `google api` models as well."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"collapsed": true,
|
|
"id": "erEHenbjaYqi"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"all_generative_models = ModelCatalog().list_generative_local_models()\n",
|
|
"print(\"\\n\\nModel Catalog - load model with ModelCatalog().load_model(model_name)\")\n",
|
|
"for i, model in enumerate(all_generative_models):\n",
|
|
"\n",
|
|
" model_name = model[\"model_name\"]\n",
|
|
" model_family = model[\"model_family\"]\n",
|
|
"\n",
|
|
" print(\"model: \", i, model)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "tLCuxZcYdTHn"
|
|
},
|
|
"source": [
|
|
"## Slim Models\n",
|
|
"Slim models are 'Function Calling' Models that perform a specialized task and output python dictionaries\n",
|
|
"- by design, slim models are specialists that **perform single function**.\n",
|
|
"- by design, slim models generally **do not require any specific** `'prompt instructions'`, but will often accept a `\"parameter\"` which is passed to the function."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"collapsed": true,
|
|
"id": "1ZS2wo8zdDOd"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"model = ModelCatalog().load_model(\"slim-sentiment-tool\")\n",
|
|
"response = model.function_call(\"That was the worst earnings call ever - what a disaster.\")\n",
|
|
"\n",
|
|
"# the 'overall' model response is just a python dictionary\n",
|
|
"print(\"\\nresponse: \", response)\n",
|
|
"print(\"llm_response: \", response['llm_response'])\n",
|
|
"print(\"sentiment: \", response['llm_response']['sentiment'])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "Ohp-shGjkDkz"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# here is one of the slim model applied against a common earnings extract\n",
|
|
"\n",
|
|
"text_passage = (\"Here’s what Costco reported for its fiscal second quarter of 2024 compared with what Wall Street \"\n",
|
|
" \"was expecting, based on a survey of analysts by LSEG, formerly known as Refinitiv: Earnings \"\n",
|
|
" \"per share: $3.92 vs. $3.62 expected. Revenue: $58.44 billion vs. $59.16 billion expected \"\n",
|
|
" \"In the three-month period that ended Feb. 18, Costco’s net income rose to $1.74 billion, or \"\n",
|
|
" \"$3.92 per share, compared with $1.47 billion, or $3.30 per share, a year earlier. \")\n",
|
|
"\n",
|
|
"# extract model takes a 'key' for a parameter, and looks for the 'value' in the text\n",
|
|
"model = ModelCatalog().load_model(\"slim-extract-tool\")\n",
|
|
"\n",
|
|
"# the general structure of a function call includes a text passage input, a function and parameters\n",
|
|
"response = model.function_call(text_passage,function=\"extract\",params=[\"revenue\"])\n",
|
|
"\n",
|
|
"print(\"\\nextract response: \", response)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "w13LLW_OdxCm"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Function calling models generally come with a test set that is a great way to learn how they work\n",
|
|
"# please note that each test can take a few minutes with 20-40 test questions\n",
|
|
"\n",
|
|
"# You can try rest of them yourself by removing comment(s) of the below catalog.\n",
|
|
"ModelCatalog().tool_test_run(\"slim-topics-tool\")\n",
|
|
"ModelCatalog().tool_test_run(\"slim-tags-tool\")\n",
|
|
"ModelCatalog().tool_test_run(\"slim-emotions-tool\")\n",
|
|
"ModelCatalog().tool_test_run(\"slim-summary-tool\")\n",
|
|
"ModelCatalog().tool_test_run(\"slim-xsum-tool\")\n",
|
|
"ModelCatalog().tool_test_run(\"slim-boolean-tool\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "kOPly8bfdnan"
|
|
},
|
|
"source": [
|
|
"## Agentic AI\n",
|
|
"Function calling models can be integrated into Agent processes which can orchestrate processes comprising multiple models and steps - most of our use cases will use the function calling models in that context\n",
|
|
"\n",
|
|
"## Last note:\n",
|
|
"Most of the models are packaged as `\"gguf\"` usually identified as GGUFGenerativeModel, or with `'-gguf'` or `'-tool` at the end of their name. These models are optimized to run most efficiently on a CPU-based laptop (especially Mac OS). You can also try the standard Pytorch versions of these models, which should yield virtually identical results, but will be slower."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "rvLVgWYMe6RO"
|
|
},
|
|
"source": [
|
|
"## Journey is yet to start!\n",
|
|
"Loved it?? This is just an example of our models. Please check out our other Agentic AI examples with every model in detail here: https://github.com/llmware-ai/llmware/tree/main/fast_start/agents\n",
|
|
"\n",
|
|
"Also, if you have more interest in RAG, then please go with our RAG examples, which you can find here: https://github.com/llmware-ai/llmware/tree/main/fast_start/rag\n",
|
|
"\n",
|
|
"If you liked it, then please **star our repo https://github.com/llmware-ai/llmware** ⭐\n",
|
|
"\n",
|
|
"Any doubts?? Join our **discord server: https://discord.gg/GN49aWx2H3** 🫂"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"colab": {
|
|
"provenance": []
|
|
},
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"name": "python"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 0
|
|
}
|