portkey-ai--gateway
wehub-resource-sync
1 行
467 KiB
Plaintext
被供应的
1 行
467 KiB
Plaintext
被供应的
{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"provenance":[]},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"}},"cells":[{"cell_type":"markdown","source":["<h1 align=\"center\">\n"," <a href=\"https://portkey.ai\">\n"," <img width=\"300\" src=\"https://analyticsindiamag.com/wp-content/uploads/2023/08/Logo-on-white-background.png\" alt=\"portkey\">\n"," </a>\n","</h1>"],"metadata":{"id":"bFj2ZAU6BtiR"}},{"cell_type":"markdown","source":["[](https://colab.research.google.com/drive/1SiyWV8ER-Gp2GEkMr9aA3KhebdeEJHBK?usp=sharing)"],"metadata":{"id":"5Mfpp2FLAYEA"}},{"cell_type":"markdown","source":["# Portkey + DeepInfra"],"metadata":{"id":"ekkvItFsWyQL"}},{"cell_type":"markdown","source":["[Portkey](https://app.portkey.ai/) is the Control Panel for AI apps. With it's popular AI Gateway and Observability Suite, hundreds of teams ship reliable, cost-efficient, and fast apps.\n","\n","With Portkey, you can\n","\n"," - Connect to 150+ models through a unified API,\n"," - View 40+ metrics & logs for all requests,\n"," - Enable semantic cache to reduce latency & costs,\n"," - Implement automatic retries & fallbacks for failed requests,\n"," - Add custom tags to requests for better tracking and analysis and more.\n"],"metadata":{"id":"x8RrtT5yYEev"}},{"cell_type":"markdown","source":["## Quickstart\n","\n","Since Portkey is fully compatible with the OpenAI signature, you can connect to the Portkey AI Gateway through OpenAI Client.\n","\n","- Set the `base_url` as `PORTKEY_GATEWAY_URL`\n","- Add `default_headers` to consume the headers needed by Portkey using the `createHeaders` helper method."],"metadata":{"id":"n1kPDnimZIXb"}},{"cell_type":"markdown","source":["You will need Portkey and Deepinfra API keys to run this notebook.\n","\n","- Sign up for Portkey and generate your API key [here](https://app.portkey.ai/).\n","- Get your Deepinfra key [here](https://deepinfra.com/dash/api_keys)\n"],"metadata":{"id":"rQSrrUBwkmOP"}},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"fffx7Tc2ghTR","executionInfo":{"status":"ok","timestamp":1714220024728,"user_tz":-330,"elapsed":20046,"user":{"displayName":"Satvik Paramkusham","userId":"09992778153373457651"}},"outputId":"09ca4f38-4ad3-415d-ac40-94b92377230f"},"outputs":[{"output_type":"stream","name":"stdout","text":["\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m62.3/62.3 kB\u001b[0m \u001b[31m1.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m311.6/311.6 kB\u001b[0m \u001b[31m9.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.6/75.6 kB\u001b[0m \u001b[31m7.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m12.7/12.7 MB\u001b[0m \u001b[31m35.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.9/77.9 kB\u001b[0m \u001b[31m5.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m2.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25h"]}],"source":["!pip install -qU portkey-ai openai"]},{"cell_type":"markdown","source":["## With OpenAI Client"],"metadata":{"id":"ptP4L78HlBUL"}},{"cell_type":"code","source":["from openai import OpenAI\n","from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders\n","from google.colab import userdata\n","\n","client = OpenAI(\n"," api_key= userdata.get('DEEPINFRA_API_KEY'), ## replace it your Mistral API key\n"," base_url=PORTKEY_GATEWAY_URL,\n"," default_headers=createHeaders(\n"," provider=\"deepinfra\",\n"," api_key= userdata.get('PORTKEY_API_KEY'), ## replace it your Portkey API key\n"," )\n",")\n","\n","chat_complete = client.chat.completions.create(\n"," model=\"meta-llama/Meta-Llama-3-70B-Instruct\",\n"," messages=[{\"role\": \"user\",\n"," \"content\": \"Who are you?\"}],\n",")\n","\n","print(chat_complete.choices[0].message.content)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"Yz7e9rokcCj0","executionInfo":{"status":"ok","timestamp":1714220347821,"user_tz":-330,"elapsed":11755,"user":{"displayName":"Satvik Paramkusham","userId":"09992778153373457651"}},"outputId":"23c67962-4166-4bb1-cd37-9ca81495a9aa"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Nice to meet you! I'm LLaMA, a large language model trained by a team of researcher at Meta AI. My primary function is to generate human-like responses to a wide range of questions and topics, from science and history to entertainment and culture.\n","\n","I'm not a human, but rather an artificial intelligence designed to simulate conversation and answer questions to the best of my knowledge. I've been trained on a massive dataset of text from the internet and can respond in multiple languages.\n","\n","I can help with things like:\n","\n","* Answering questions on a variety of topics\n","* Generating text on a given topic or subject\n","* Translating text from one language to another\n","* Summarizing long pieces of text into shorter, more digestible versions\n","* Offering suggestions or ideas for creative projects\n","* Even just having a conversation and chatting about your day or interests!\n","\n","So, what's on your mind? Want to chat about something specific or just see where the conversation takes us?\n"]}]},{"cell_type":"markdown","source":["## Observability with Portkey"],"metadata":{"id":"wMhOOkaLqkSp"}},{"cell_type":"markdown","source":["By routing requests through Portkey you can track a number of metrics like - tokens used, latency, cost, etc.\n","\n","Here's a screenshot of the dashboard you get with Portkey!"],"metadata":{"id":"LzECvXQaqr6Z"}},{"cell_type":"markdown","source":["\n"],"metadata":{"id":"W80YWspqr7s0"}},{"cell_type":"code","source":[],"metadata":{"id":"yqrIH5mY01BI"},"execution_count":null,"outputs":[]}]} |