{ "cells": [ { "cell_type": "code", "execution_count": null, "metadata": { "cellView": "form", "id": "2RexaI2BoZZy" }, "outputs": [], "source": [ "# Copyright 2026 Google LLC\n", "#\n", "# Licensed under the Apache License, Version 2.0 (the \"License\");\n", "# you may not use this file except in compliance with the License.\n", "# You may obtain a copy of the License at\n", "#\n", "# https://www.apache.org/licenses/LICENSE-2.0\n", "#\n", "# Unless required by applicable law or agreed to in writing, software\n", "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", "# See the License for the specific language governing permissions and\n", "# limitations under the License." ] }, { "cell_type": "markdown", "metadata": { "id": "rJ43O7oc6xGP" }, "source": [ "# Configuring Retries in the Gen AI SDK\n", "\n", "\n", " \n", " \n", " \n", " \n", "
\n", " \n", " \"Google
Open in Colab\n", " \n", " 		\n", " \n", " \"Google
Open in Colab Enterprise\n", " \n", " 		\n", " \n", " \"Workbench
Open in Workbench\n", " \n", " 		\n", " \n", " \"GitHub
View on GitHub\n", " \n", "
\n", "\n", "
\n", "\n", "

\n", "Share to:\n", "\n", "\n", " \"LinkedIn\n", "\n", "\n", "\n", " \"Bluesky\n", "\n", "\n", "\n", " \"X\n", "\n", "\n", "\n", " \"Reddit\n", "\n", "\n", "\n", " \"Facebook\n", "\n", "

" ] }, { "cell_type": "markdown", "metadata": { "id": "uYusEvgifp9U" }, "source": [ "| Author |\n", "| --- |\n", "| [Eric Dong](https://github.com/gericdong) |" ] }, { "cell_type": "markdown", "metadata": { "id": "8sg1R6UQ7Q3t" }, "source": [ "## Overview\n", "\n", "When interacting with Gemini through Gemini API, transient network errors or service disruptions can occur. Implementing a retry strategy is crucial for building robust and reliable applications. This guide explains how to configure HTTP retry options for the [Google Gen AI SDK](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/sdks/overview).\n", "\n", "\n", "### **Why Configure Retries?**\n", "\n", "By default, the Gen AI SDK has some level of retry for certain types of errors. However, for fine-grained control over the retry behavior, you can configure your own retry strategy. This is particularly important for:\n", "\n", "- **Handling transient errors**: Automatically retry requests that fail due to temporary issues, such as network timeouts or temporary server unavailability (e.g., 5xx errors).\n", "- **Managing rate limits**: If you anticipate hitting rate limits (e.g., 429 errors), a retry strategy with exponential backoff can help manage the request rate and avoid overwhelming the service.\n", "- **Improving application resilience**: Make your application more resilient to intermittent issues, leading to a better user experience.\n" ] }, { "cell_type": "markdown", "metadata": { "id": "gPiTOAHURvTM" }, "source": [ "## Getting Started" ] }, { "cell_type": "markdown", "metadata": { "id": "-tn3uw268iw4" }, "source": [ "### Install Google Gen AI SDK for Python\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "3CaXL22k8iw4" }, "outputs": [], "source": [ "%pip install --upgrade --quiet google-genai" ] }, { "cell_type": "markdown", "metadata": { "id": "xW5WwfAOfp9V" }, "source": [ "### Import libraries\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "o0JUCkvVfp9V" }, "outputs": [], "source": [ "import os\n", "import sys\n", "\n", "from IPython.display import Markdown, display\n", "from google import genai\n", "from google.genai import types" ] }, { "cell_type": "markdown", "metadata": { "id": "SY-GRP3m8iw5" }, "source": [ "### Authenticate your notebook environment\n", "\n", "If you are running this notebook on Google Colab, run the cell below to authenticate your environment." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "06RAe75C8iw5" }, "outputs": [], "source": [ "if \"google.colab\" in sys.modules:\n", " from google.colab import auth\n", "\n", " auth.authenticate_user()" ] }, { "cell_type": "markdown", "metadata": { "id": "wkInCpT9fp9V" }, "source": [ "### Authenticate your Google Cloud Project for Vertex AI\n", "\n", "You can use a Google Cloud Project or an API Key for authentication. This tutorial uses a Google Cloud Project.\n", "\n", "- [Enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "z-jVOPQVfp9V" }, "outputs": [], "source": [ "# fmt: off\n", "PROJECT_ID = \"[your-project-id]\" # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n", "# fmt: on\n", "if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n", " PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n", "\n", "LOCATION = \"global\"" ] }, { "cell_type": "markdown", "metadata": { "id": "LBGTO9MUFWP0" }, "source": [ "## **Option 1**: Configuring Retries at the Client Level\n", "\n", "The Gen AI SDK provides the `genai.types.HttpRetryOptions` class to configure a custom retry policy. You can then apply this policy to the `genai.Client` using `genai.types.HttpOptions`." ] }, { "cell_type": "markdown", "metadata": { "id": "RR5RO9KxFvRR" }, "source": [ "### **Step 1**: Define `HttpRetryOptions`\n", "\n", "First, create an instance of `HttpRetryOptions` and specify the desired retry behavior. Here are some of the key parameters:\n", "\n", "- `initial_delay`: The initial delay in seconds before the first retry.\n", "- `attempts`: The maximum number of retry attempts.\n", "- `exp_base`: The base for the exponential backoff calculation.\n", "- `max_delay`: The maximum delay in seconds between retries.\n", "- `jitter`: A factor to add a random delay to the backoff, which can help avoid a \"thundering herd\" problem.\n", "- `http_status_codes`: A list of HTTP status codes that should trigger a retry." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "pzHKE306dSJ2" }, "outputs": [], "source": [ "# Configure the retry policy.\n", "retry_options = types.HttpRetryOptions(\n", " initial_delay=1.0, # Initial delay of 1 second.\n", " attempts=5, # Retry up to 5 times.\n", " exp_base=2, # Double the delay for each subsequent retry.\n", " http_status_codes=[429, 500, 502, 503, 504], # Retry on these server errors.\n", ")" ] }, { "cell_type": "markdown", "metadata": { "id": "sP0bwwcoGJKE" }, "source": [ "### **Step 2**: Apply to the Client with `HttpOptions`\n", "\n", "Next, create an instance of `HttpOptions` and pass your `retry_options` to it.\n", "\n", "You can also configure other HTTP options, such as `timeout` - Timeout for the request in milliseconds." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "D90UFQOpGaEh" }, "outputs": [], "source": [ "# Configure HTTP options with the retry settings.\n", "http_options = types.HttpOptions(\n", " retry_options=retry_options,\n", " timeout=120 * 1000, # Set a 120-second timeout for requests.\n", ")" ] }, { "cell_type": "markdown", "metadata": { "id": "u8Vh9ePQGp_A" }, "source": [ "### **Step 3**: Initialize the `genai.Client`\n", "\n", "Finally, initialize the `genai.Client` with the `http_options`." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "lSvDL3LoGzDx" }, "outputs": [], "source": [ "# Initialize the client with the configured HTTP options.\n", "client = genai.Client(\n", " vertexai=True,\n", " project=PROJECT_ID,\n", " location=LOCATION,\n", " http_options=http_options,\n", ")" ] }, { "cell_type": "markdown", "metadata": { "id": "k5aRhD50HBSM" }, "source": [ "Now, any API calls made with this client will use the configured retry policy." ] }, { "cell_type": "markdown", "metadata": { "id": "5kFIrV5NfD0d" }, "source": [ "### Complete Code Example\n", "\n", "Here is the complete code sample." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "pdoPLP6lfMFH" }, "outputs": [], "source": [ "client = genai.Client(\n", " vertexai=True,\n", " project=PROJECT_ID,\n", " location=LOCATION,\n", " http_options=types.HttpOptions(\n", " retry_options=types.HttpRetryOptions(\n", " initial_delay=1.0,\n", " attempts=10,\n", " http_status_codes=[429, 500, 502, 503, 504],\n", " ),\n", " timeout=120 * 1000,\n", " ),\n", ")\n", "\n", "# You can now use this client to make API calls with the custom retry policy.\n", "# For example:\n", "response = client.models.generate_content(\n", " model=\"gemini-3.5-flash\", contents=\"Tell me a story\"\n", ")\n", "\n", "display(Markdown(response.text))" ] }, { "cell_type": "markdown", "metadata": { "id": "b5386769b966" }, "source": [ "## **Option 2**: Configuring Retries at the Request Level\n", "\n", "You can also set `HttpRetryOptions` per-request using the `config` parameter." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "620c8177fc44" }, "outputs": [], "source": [ "response = client.models.generate_content(\n", " model=\"gemini-3.5-flash\",\n", " contents=\"Tell me a joke about a rabbit.\",\n", " config=types.GenerateContentConfig(\n", " http_options=types.HttpOptions(\n", " retry_options=types.HttpRetryOptions(\n", " initial_delay=1.0,\n", " attempts=10,\n", " http_status_codes=[429, 500, 502, 503, 504],\n", " ),\n", " timeout=120 * 1000,\n", " )\n", " ),\n", ")\n", "\n", "display(Markdown(response.text))" ] } ], "metadata": { "colab": { "name": "configure_retries.ipynb", "toc_visible": true }, "kernelspec": { "display_name": "Python 3", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 0 }