deepset-ai--haystack
wehub-resource-sync
c56bef871b
CodeQL / Analyze (python) (push) Has been cancelled
Update Platform Components Table / update (push) Has been cancelled
Docker image release / Build base image (push) Has been cancelled
Sync docs with Docusaurus / sync (push) Has been cancelled
Tests / Check if changed (push) Has been cancelled
Tests / format (push) Has been cancelled
Tests / check-imports (push) Has been cancelled
Tests / Unit / macos-latest (push) Has been cancelled
Tests / Unit / ubuntu-latest (push) Has been cancelled
Tests / Unit / windows-latest (push) Has been cancelled
Tests / mypy (push) Has been cancelled
Tests / Integration / ubuntu-latest (push) Has been cancelled
Tests / Integration / macos-latest (push) Has been cancelled
Tests / Integration / windows-latest (push) Has been cancelled
Tests / notify-slack-on-failure (push) Has been cancelled
Tests / Mark tests as completed (push) Has been cancelled
80 行
2.5 KiB
Plaintext
80 行
2.5 KiB
Plaintext
---
|
|
title: "VertexAIImageQA"
|
|
id: vertexaiimageqa
|
|
slug: "/vertexaiimageqa"
|
|
description: "This component enables text generation (image captioning) using Google Vertex AI generative models."
|
|
---
|
|
|
|
# VertexAIImageQA
|
|
|
|
This component enables text generation (image captioning) using Google Vertex AI generative models.
|
|
|
|
<div className="key-value-table">
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| **Mandatory run variables** | `image`: A [`ByteStream`](../../concepts/data-classes.mdx#bytestream) containing an image data <br /> <br />`question`: A string of a question about the image |
|
|
| **Output variables** | `replies`: A list of strings containing answers generated by the model |
|
|
| **API reference** | [Google Vertex](/reference/integrations-google-vertex) |
|
|
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/google_vertex |
|
|
|
|
</div>
|
|
|
|
`VertexAIImageQA` supports the `imagetext` model.
|
|
|
|
### Parameters Overview
|
|
|
|
`VertexAIImageQA` uses Google Cloud Application Default Credentials (ADCs) for authentication. For more information on how to set up ADCs, see the [official documentation](https://cloud.google.com/docs/authentication/provide-credentials-adc).
|
|
|
|
Keep in mind that it’s essential to use an account that has access to a project authorized to use Google Vertex AI endpoints.
|
|
|
|
You can find your project ID in the [GCP resource manager](https://console.cloud.google.com/cloud-resource-manager) or locally by running `gcloud projects list` in your terminal. For more info on the gcloud CLI, see its [official documentation](https://cloud.google.com/cli).
|
|
|
|
## Usage
|
|
|
|
You need to install `google-vertex-haystack` package to use the `VertexAIImageQA`:
|
|
|
|
```python
|
|
pip install google-vertex-haystack
|
|
```
|
|
|
|
### On its own
|
|
|
|
Basic usage:
|
|
|
|
```python
|
|
from haystack.dataclasses.byte_stream import ByteStream
|
|
from haystack_integrations.components.generators.google_vertex import VertexAIImageQA
|
|
|
|
qa = VertexAIImageQA()
|
|
|
|
image = ByteStream.from_file_path("dog.jpg")
|
|
|
|
res = qa.run(image=image, question="What color is this dog")
|
|
|
|
print(res["replies"][0])
|
|
|
|
>>> white
|
|
```
|
|
|
|
You can also set the number of answers generated:
|
|
|
|
```python
|
|
from haystack.dataclasses.byte_stream import ByteStream
|
|
from haystack_integrations.components.generators.google_vertex import VertexAIImageQA
|
|
|
|
qa = VertexAIImageQA(
|
|
number_of_results=3,
|
|
)
|
|
image = ByteStream.from_file_path("dog.jpg")
|
|
|
|
res = qa.run(image=image, question="Tell me something about this dog")
|
|
|
|
for answer in res["replies"]:
|
|
print(answer)
|
|
|
|
>>> pomeranian
|
|
>>> white
|
|
>>> pomeranian puppy
|
|
```
|