microsoft--promptflow
wehub-resource-sync
e768098d0e
tools_continuous_delivery / Private PyPI non-main branch release (push) Has been skipped
tools_continuous_delivery / Private PyPI main branch release (push) Failing after 2m42s
Publish Promptflow Doc / Build (push) Has been cancelled
Publish Promptflow Doc / Deploy (push) Has been cancelled
Flake8 Lint / flake8 (push) Has been cancelled
Spell check CI / Spell_Check (push) Has been cancelled
359 行
9.4 KiB
Plaintext
359 行
9.4 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Prompty output format"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"**Learning Objectives** - Upon completing this tutorial, you should be able to:\n",
|
|
"\n",
|
|
"- Understand how to handle output format of prompty like: text, json_object.\n",
|
|
"- Understand how to consume stream output of prompty\n",
|
|
"\n",
|
|
"## 0. Install dependent packages"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%capture --no-stderr\n",
|
|
"%pip install promptflow-devkit"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## 1. Create necessary connections\n",
|
|
"Connection helps securely store and manage secret keys or other sensitive credentials required for interacting with LLM and other external tools for example Azure Content Safety.\n",
|
|
"\n",
|
|
"Above prompty uses connection `open_ai_connection` inside, we need to set up the connection if we haven't added it before. After created, it's stored in local db and can be used in any flow.\n",
|
|
"\n",
|
|
"Prepare your Azure OpenAI resource follow this [instruction](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal) and get your `api_key` if you don't have one."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from promptflow.client import PFClient\n",
|
|
"from promptflow.connections import AzureOpenAIConnection, OpenAIConnection\n",
|
|
"\n",
|
|
"# client can help manage your runs and connections.\n",
|
|
"pf = PFClient()\n",
|
|
"try:\n",
|
|
" conn_name = \"open_ai_connection\"\n",
|
|
" conn = pf.connections.get(name=conn_name)\n",
|
|
" print(\"using existing connection\")\n",
|
|
"except:\n",
|
|
" # Follow https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/create-resource?pivots=web-portal to create an Azure OpenAI resource.\n",
|
|
" connection = AzureOpenAIConnection(\n",
|
|
" name=conn_name,\n",
|
|
" api_key=\"<your_AOAI_key>\",\n",
|
|
" api_base=\"<your_AOAI_endpoint>\",\n",
|
|
" api_type=\"azure\",\n",
|
|
" )\n",
|
|
"\n",
|
|
" # use this if you have an existing OpenAI account\n",
|
|
" # connection = OpenAIConnection(\n",
|
|
" # name=conn_name,\n",
|
|
" # api_key=\"<user-input>\",\n",
|
|
" # )\n",
|
|
"\n",
|
|
" conn = pf.connections.create_or_update(connection)\n",
|
|
" print(\"successfully created connection\")\n",
|
|
"\n",
|
|
"print(conn)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## 2. Format prompty output"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Text output\n",
|
|
"By default the prompty returns the message of first choices."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"with open(\"text_format.prompty\") as fin:\n",
|
|
" print(fin.read())"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from promptflow.core import Prompty\n",
|
|
"\n",
|
|
"# load prompty as a flow\n",
|
|
"f = Prompty.load(\"text_format.prompty\")\n",
|
|
"# execute the flow as function\n",
|
|
"question = \"What is the capital of France?\"\n",
|
|
"result = f(first_name=\"John\", last_name=\"Doe\", question=question)\n",
|
|
"\n",
|
|
"# note: the result is a string\n",
|
|
"result"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Json object output\n",
|
|
"\n",
|
|
"When the user meets the following conditions, prompty returns content of first choices as a dict.\n",
|
|
"- Define `response_format` to `type: json_object` in parameters \n",
|
|
"- Specify the return json format in template.\n",
|
|
"\n",
|
|
"Note: response_format is compatible with GPT-4 Turbo and all GPT-3.5 Turbo models newer than gpt-3.5-turbo-1106. For more details, refer to this [document](https://platform.openai.com/docs/api-reference/chat/create#chat-create-response_format)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"with open(\"json_format.prompty\") as fin:\n",
|
|
" print(fin.read())"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from promptflow.core import Prompty\n",
|
|
"\n",
|
|
"# load prompty as a flow\n",
|
|
"f = Prompty.load(\"json_format.prompty\")\n",
|
|
"# execute the flow as function\n",
|
|
"question = \"What is the capital of France?\"\n",
|
|
"result = f(first_name=\"John\", last_name=\"Doe\", question=question)\n",
|
|
"\n",
|
|
"# note: the result is a dict\n",
|
|
"result"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### All choices\n",
|
|
"\n",
|
|
"When the user configures response as `all`, prompty will return the raw LLM response which has all the choices.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"with open(\"all_response.prompty\") as fin:\n",
|
|
" print(fin.read())"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from promptflow.core import Prompty\n",
|
|
"\n",
|
|
"# load prompty as a flow\n",
|
|
"f = Prompty.load(\"all_response.prompty\")\n",
|
|
"# execute the flow as function\n",
|
|
"question = \"What is the capital of France?\"\n",
|
|
"result = f(first_name=\"John\", last_name=\"Doe\", question=question)\n",
|
|
"\n",
|
|
"# note: the result is a ChatCompletion object\n",
|
|
"print(result.choices[0])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Streaming output"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"When `stream=true` is configured in the parameters of a prompt whose output format is text, promptflow sdk will return a generator type, which item is the content of each chunk."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"with open(\"stream_output.prompty\") as fin:\n",
|
|
" print(fin.read())"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from promptflow.core import Prompty\n",
|
|
"\n",
|
|
"# load prompty as a flow\n",
|
|
"f = Prompty.load(\"stream_output.prompty\")\n",
|
|
"# execute the flow as function\n",
|
|
"question = \"What's the steps to get rich?\"\n",
|
|
"result = f(question=question)\n",
|
|
"for item in result:\n",
|
|
" print(item, end=\"\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Notes: When `stream=True`, if the response format is `json_object` or response is `all`, LLM response will be returned directly. For more details about handle stream response, refer to this [document](https://platform.openai.com/docs/api-reference/chat/create#chat-create-stream).\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Batch run with text output"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from promptflow.client import PFClient\n",
|
|
"\n",
|
|
"data = \"./data.jsonl\" # path to the data file\n",
|
|
"\n",
|
|
"# create run with the flow and data\n",
|
|
"pf = PFClient()\n",
|
|
"base_run = pf.run(\n",
|
|
" flow=\"text_format.prompty\",\n",
|
|
" data=data,\n",
|
|
" column_mapping={\n",
|
|
" \"question\": \"${data.question}\",\n",
|
|
" },\n",
|
|
" stream=True,\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"details = pf.get_details(base_run)\n",
|
|
"details.head(10)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Batch run with stream output"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from promptflow.client import PFClient\n",
|
|
"\n",
|
|
"data = \"./data.jsonl\" # path to the data file\n",
|
|
"\n",
|
|
"# create run with the flow and data\n",
|
|
"pf = PFClient()\n",
|
|
"base_run = pf.run(\n",
|
|
" flow=\"stream_output.prompty\",\n",
|
|
" data=data,\n",
|
|
" column_mapping={\n",
|
|
" \"question\": \"${data.question}\",\n",
|
|
" },\n",
|
|
" stream=True,\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"details = pf.get_details(base_run)\n",
|
|
"details.head(10)"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"build_doc": {
|
|
"author": [
|
|
"lalala123123@github.com",
|
|
"wangchao1230@github.com"
|
|
],
|
|
"category": "local",
|
|
"section": "Prompty",
|
|
"weight": 30
|
|
},
|
|
"kernelspec": {
|
|
"display_name": "prompt",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.9.18"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
}
|