Encountering 503 Error When Calling Gemini API from Google Colab

Question

I'm working on a project using Google Colab to run Python code that interacts with the Gemini API (a part of Google's Cloud AI tools). The goal is to automate call transcript categorization into predefined categories using Gemini's AI.

Here's a brief overview of what I'm doing: I read an Excel file of call transcripts, send these transcripts to Gemini for categorization, and then update the Excel file based on the categories identified by the AI (marking them with 0s and 1s).

Below is a snippet of my code for setting up the API and sending a request to Gemini:

import google.generativeai as genai

GOOGLE_API_KEY = "your_api_key_here"
genai.configure(api_key=GOOGLE_API_KEY)
model = genai.GenerativeModel('gemini-pro')

def send_to_gemini(transcript):
    prompt = f"Categorize the following transcript: {transcript}"
    try:
        response = model.generate_content(prompt)
        return response.text
    except Exception as e:
        print(f"Failed to send request to Gemini: {e}")

However, I keep getting an ERROR:tornado.access:503 suggesting a server-side issue:

ERROR:tornado.access:503 POST /v1beta/models/gemini-pro:generateContent (127.0.0.1) 4039.47ms

Any advice or insights would be greatly appreciated.

can i see how you are converting the contents of the excel file to a string. are you coverting it to a csv or something? — Linda Lawton - DaImTo
– Linda Lawton - DaImTo, Commented Mar 13, 2024 at 13:47
@LindaLawton-DaImTo In my code the Excel file's contents are not being converted to a string or CSV directly. Instead, the pd.read_excel(path_excel_input) function from the pandas library is used to read the Excel file into a DataFrame. The transcription and category descriptions are assembled into a prompt string within the Python script. This promptis sent to Gemini through the API. — nils
– nils, Commented Mar 14, 2024 at 9:57
wow never tried that generate_content only takes string so wonder what it will think of that format. can you do a print (prompt) wonder what that looks like — Linda Lawton - DaImTo
– Linda Lawton - DaImTo, Commented Mar 14, 2024 at 10:04

Prisoner · Accepted Answer · 2024-03-13 14:36:01Z

17

Error 503 corresponds to "service unavailable".

If you read the full error message on the response, you might get "The model is overloaded. Please try again later."

This is definitely something going on on Google's end - not yours. There is very little you can do about it. However, you should certainly account for it. Retrying attempts in a loop with progressive back-off is a standard approach to server unavailable messages.

You indicate you're using the "google.generativeai" package, which corresponds with the AI Studio offering (aistudio.google.com) - not Google Cloud. This is currently a free-tier only service that is not Generally Available. So it seems likely they are still tuning things to meet expected user demand and scale things out as they get ready for production and for the introduction of an additional paid tier.

edited Mar 13, 2024 at 14:36

answered Mar 13, 2024 at 14:15

Prisoner

50.9k7 gold badges59 silver badges113 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Linda Lawton - DaImTo Over a year ago

What i dont get is im not really seeing this.

Utku Over a year ago

I know google has not offer this product with pay or not guarantee %99.99 uptime. Do you know any thing how we should avoid this situation?

ericOnline Over a year ago

I'm getting 503 recvmsg:Connection reset by peer so sometimes, the 503 has a different error message.

Prisoner Over a year ago

@ericOnline - there is now both a free tier and paid tier. An occasional error 503 is not uncommon, but if you're seeing it frequently, you should probably open a question with the details about what you're doing and seeing.

ericOnline Over a year ago

Unfortunately, from my tests, the paid version of Google AI Studio API does not like token counts > 1M. I frequently received Status 429, 503 and 500 return codes. I changed to Vertex AI and the same code consistently gives Status 200.

|

codeananda · Accepted Answer · 2025-02-03 11:20:10Z

2

This isn't exactly answering the question but will be helpful for those who, like me, found this page because they were getting a ton of 503 errors when using the google.generativeai package

I had this error when caching pdfs.

Switching to vertexai helped and removed most of the errors instantly.

Instead of using google.generativeai's CachedContent, use vertexai's implementation, for example. There will be a bit of fiddling but not a lot. Plus, it uses the same funds from your google account, so no more billing to set up.

edited Feb 3 at 11:20

answered Jan 30 at 14:38

codeananda

1,4482 gold badges19 silver badges27 bronze badges

2 Comments

truongnm Sep 5 at 23:14

thanks! At lease can you link a docs to vertexai could be even more helpful. Thanks again

msersen Sep 9 at 0:09

I've been writing an app landsurv.ai using aistudio and I too get these errors, mostly with PDF's. How can I switch to vertexai's implementation?

Collectives™ on Stack Overflow

Encountering 503 Error When Calling Gemini API from Google Colab

2 Answers 2

7 Comments

2 Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

7 Comments

2 Comments

Related