microsoft--promptflow
wehub-resource-sync
e768098d0e
tools_continuous_delivery / Private PyPI non-main branch release (push) Has been skipped
tools_continuous_delivery / Private PyPI main branch release (push) Failing after 2m42s
Publish Promptflow Doc / Build (push) Has been cancelled
Publish Promptflow Doc / Deploy (push) Has been cancelled
Flake8 Lint / flake8 (push) Has been cancelled
Spell check CI / Spell_Check (push) Has been cancelled
39 行
2.2 KiB
Markdown
39 行
2.2 KiB
Markdown
# LLMLingua Prompt Compression
|
|
|
|
## Introduction
|
|
LLMLingua Prompt Compression tool enables you to speed up large language model's inference and enhance large language model's perceive of key information, compress the prompt with minimal performance loss.
|
|
|
|
## Requirements
|
|
PyPI package: [`llmlingua-promptflow`](https://pypi.org/project/llmlingua-promptflow/).
|
|
- For Azure users:
|
|
follow [the wiki for AzureML](https://learn.microsoft.com/en-us/azure/machine-learning/prompt-flow/how-to-custom-tool-package-creation-and-usage?view=azureml-api-2#prepare-compute-session) or [the wiki for AI Studio](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/prompt-flow-tools/prompt-flow-tools-overview#custom-tools) to prepare the compute session.
|
|
- For local users:
|
|
```
|
|
pip install llmlingua-promptflow
|
|
```
|
|
You may also want to install the [Prompt flow for VS Code extension](https://marketplace.visualstudio.com/items?itemName=prompt-flow.prompt-flow).
|
|
|
|
## Prerequisite
|
|
Create a MaaS deployment for large language model in Azure model catalog. Take the Llama model as an example, you can learn how to deploy and consume Meta Llama models with model as a service by [the guidance for Azure AI Studio](https://learn.microsoft.com/azure/ai-studio/how-to/deploy-models-llama?tabs=llama-three).
|
|
|
|
## Inputs
|
|
|
|
The tool accepts the following inputs:
|
|
|
|
| Name | Type | Description | Required |
|
|
| ---- | ---- | ----------- | -------- |
|
|
| prompt | string | The prompt that needs to be compressed. | Yes |
|
|
| myconn | CustomConnection | The created connection to a MaaS resource for calculating log probability. | Yes |
|
|
| rate | float | The maximum compression rate target to be achieved. Default value is 0.5. | No |
|
|
|
|
## Outputs
|
|
|
|
| Return Type | Description |
|
|
|-------------|----------------------------------------------------------------------|
|
|
| string | The resulting compressed prompt. |
|
|
|
|
## Sample Flows
|
|
Find example flows using the `llmlingua-promptflow` package [here](https://github.com/microsoft/promptflow/tree/main/examples/flows/integrations/llmlingua-prompt-compression).
|
|
|
|
## Contact
|
|
Please reach out to LLMLingua Team (<llmlingua@microsoft.com>) with any issues. |