Your AI gateway holds every provider key. Treat it that way.

What happened on Wednesday

LiteLLM is a popular open-source gateway. Your app sends it one kind of request, and it forwards the call to OpenAI, Anthropic, Google or whichever provider you've configured, while tracking spend and handing out its own "virtual" keys to your services. Plenty of AI features built in the last two years run through it or something like it.

On October 21, 2026, a malicious version, 1.107.0, was published to PyPI and pushed to LiteLLM's Docker images. According to LiteLLM's incident post, the package was live on PyPI from 7:39 to 9:23 AM ET, and the latest, main-latest, main-stable and v1.107.0 Docker tags pointed at the bad image until about 10:20 AM ET. Anyone who installed it unpinned, or restarted a container on one of those tags in that window, should assume they were hit.

The payload was nasty in the ordinary ways: it ran whenever Python started, including when you ran pip to fix it, and it collected cloud credentials, SSH keys and browser data. But one detail from Wiz's analysis stands out. LiteLLM's proxy stores the upstream provider keys in a database, and the stealer queried that table directly to pull out every configured provider credential.

In other words, it went straight for the reason you run a gateway in the first place.

The gateway is the point, and the problem

A gateway is useful because it centralizes things. One place for every provider key. One place for spend limits and routing rules. One set of logs showing every prompt and every response. Your services never see the real provider keys, which is good.

But that's also exactly what makes it the most valuable box in your AI setup. Whoever gets into the gateway gets every provider at once, the ability to run up your bills on all of them, and possibly a log of what your customers typed.

Most teams treat the gateway as plumbing. It goes up in an afternoon, it works, and nobody thinks about it again. They protect the app that faces customers, and they protect the main database. The gateway sits in between with the most powerful credentials in the building and much less attention.

Convenience put every key in one table. The protection has to be just as concentrated.

The way in started at a gateway too

The part of LiteLLM's write-up worth reading twice is how the attacker got in. LiteLLM says it believes the attacker first got into its own internal LiteLLM deployment, which was exposed to the public internet and had an unexpected, innocuously named user account on it. In that deployment's logs, they found GitHub command-line tokens, because, LiteLLM says, in some situations a coding assistant's response bodies contained a developer's token and those bodies were being logged. With those tokens, the attacker got into the release workflow.

LiteLLM is still confirming parts of this, and suspects a recently patched vulnerability for the first step. But the pattern doesn't depend on the details. A gateway was reachable from the internet. It had an account nobody expected. Its logs held secrets because it logged whole request and response bodies.

Those are three settings, not three exotic attacks. Every one of them is probably worth checking on yours.

What to change

Here's how we'd set a gateway up now.

Give it its own keys. The provider keys in the gateway should be used only by the gateway, one set per environment. Staging shouldn't hold production keys. If a key leaks, you'll know where from.

Cap spend at the provider, not just the gateway. Gateway spend limits don't help if someone has the provider key itself. Set budgets and alerts in each provider's console.

Keep the admin side private. The proxy's admin interface and database shouldn't be reachable from the internet. Review the user accounts on it, and remove any you can't explain.

Decide what it logs. Logging full prompts and responses is handy for debugging and dangerous to keep. Log metadata by default, redact the rest, and keep a short retention. Our note on deciding what the agent may log goes further.

Pin it, by version and ideally by digest. If your deploy pulls latest, you'll take whatever is published next. Pin the exact version, upgrade on purpose, and read the changelog first.

If someone else runs your gateway

Not everyone self-hosts. Many teams use a hosted gateway or a router service instead, and the same logic applies, just with someone else holding the table. The questions move into the contract and the settings page.

Ask the vendor where provider keys are stored and who on their side can read them. Ask whether you can bring your own keys per environment, or whether everything sits under one account. Ask what they log by default, how long they keep it, and whether you can turn full-body logging off. Ask how they'd tell you about a compromise, and how fast. If the answers are vague, treat their gateway the way you'd treat your own unpinned one.

And whether it's hosted or yours, keep a short list of every place a provider key lives today. On most teams it's longer than anyone expects: the gateway, a couple of serverless functions, a developer's environment file, an old prototype nobody switched off.

Rehearse the rotation

LiteLLM's advice to affected users was to rotate the instance's cloud role, every provider key stored in the proxy database, the master and virtual keys, the database password and any secret in the host's environment. For many teams, that list would take a day to even write down.

Write it now, while nothing is on fire. Which provider keys does the gateway hold? Who can create new ones at each provider? Which services use virtual keys, and how do they pick up new ones? Then rotate one environment end to end and time it. Our credential rotation note has the cutover pattern, and when a package goes bad covers the first hour, including the trap of running the package manager first.

If you weren't affected on Wednesday, this is still the cheapest time to do it. Wiz says it saw stolen cloud credentials being tested about four and a half hours after they were taken. That's not much time to find out who has access to the OpenAI console.

If you'd like help reviewing how your AI features handle keys, logs and providers, that's part of the AI applications and agents work we do. Talk to us.

Related work: Sutton|Place.