r/googlecloud • u/philwinder • 15h ago
AI/ML Vertex Gemini has zero capacity outside of the US
I'm in the UK and recently I've been working on a client project where UK or EU data residency is desirable. We need gemini to do some things for us, but when using either london, EU, or global endpoints, vertex gemini is constantly giving 429 out of capacity errors.
Looking at the metrics, I can see that our request rate is very small, roughly 0.5 RPS, max of about 2. But the 429 error rate is up to 80%.
At first I thought it was a glitch. Maybe related to London's outage the other day. But this has been happening for more than a week now and continues today. It changes throughout the day, as you'd expect, but still very very high error rates.
On a previous project we never had these issues. The only difference was that this client was US-based and we were using the us endpoint (not global). So this morning I switched to that and boom, ~0.5-1% error rate.
What I don't get is why the internet isn't complaining about this? It's ridiculous, they should just turn it off until they can handle some non-zero load.
Edit: updated us estimates to 1% region. Also, I forgot to mention GCP's dynamic quota nonsense. It's impossible to estimate what your RPS quota actually is. This alone has pushed me towards OpenAI/etc. Purely because their quota system is more transparent. Again, to me, this is crazy given how much experience GCP has in this space.
