Written by Jeremy Souffir Founder, JTS Tech Services
The short version: on 24 March 2026, attackers published two backdoored releases of LiteLLM — a widely used open-source gateway that sits between applications and the various AI model providers — to PyPI, the standard Python package index. The packages were live for roughly forty minutes before removal. In that window they were pulled into build pipelines, where the payload ran automatically and harvested whatever secrets it found: AWS, Google Cloud and Azure credentials, SSH keys, Kubernetes tokens, database passwords, CI/CD secrets and LLM API keys. Research published this month puts the exposure at more than 2,500 organisations and around 434,000 CI/CD pipelines. The reason it is news again in August rather than in March is the uncomfortable half: removing a package undoes nothing. The credentials it copied are still valid wherever nobody rotated them, and the FBI has warned that the actors involved are likely to use them long after the original intrusion.
What actually happened?
The chain is worth following precisely, because the interesting part is not that a package was poisoned. It is where the poison came from.
- The attackers first compromised Trivy — a well-regarded open-source vulnerability scanner — by way of a leaked automation token, and pushed malicious code into the tags its GitHub Actions workflow resolves to
- LiteLLM ran that scanner inside its own CI/CD pipeline, as a security step. The poisoned scanner executed with the pipeline's privileges and took the project's PyPI publishing tokens
- On 24 March, using those tokens, the attackers published LiteLLM v1.82.7 and v1.82.8 to PyPI — genuine releases from the genuine account, signed off by the genuine automation
- The payload was a `.pth` file dropped into Python's site-packages. That detail matters: a `.pth` file executes at interpreter startup, so the code ran on every Python invocation on the machine, whether or not anything ever imported LiteLLM
- It collected environment variables, cloud credentials, SSH keys, Kubernetes tokens and database passwords, encrypted them, and sent them to two domains dressed up to look legitimate — one echoing the project's own name, one echoing a well-known security vendor's
- The packages were removed after about forty minutes. LiteLLM published a detailed advisory, rotated maintainer credentials and told users to pin to v1.82.6 or earlier
The detail worth sitting with
The entry point was a security tool. LiteLLM was doing the thing everybody is told to do — running a vulnerability scanner in CI — and that scanner is what carried the compromise inside. This is not an argument against scanning, and we would still tell you to scan. It is an argument against the assumption underneath most supply-chain policy: that your build pipeline's own tooling is a trusted zone. Your CI runner holds publishing tokens, cloud credentials and deploy keys, and every action, scanner and base image you invoke there runs with access to them. It is one of the highest-privilege environments you operate, and in most mid-market companies it is also the least reviewed.
We don't use LiteLLM. Does this affect us?
Possibly, and the honest answer is that most businesses cannot tell without checking. Three reasons this reaches further than the list of companies who deliberately chose this tool.
- AI middleware arrives transitively. Very few businesses sit down and select a model gateway. It comes bundled inside an AI feature your development agency built, a self-hosted tool your team deployed, an internal chatbot, or a Docker image somebody based a service on. The question is not whether you chose it — it is whether it is anywhere in your dependency tree
- The blast radius is credentials, not code. Even a business that never shipped the bad version to production may have run it once in a build. That build had secrets in its environment, and those secrets are as valid today as they were in March unless somebody rotated them
- Your exposure may sit with a supplier. If an agency, contractor or SaaS vendor builds and deploys on your behalf, their pipeline holds your cloud credentials. Their unrotated key is your open door, and it will not appear in any scan you run against your own estate
The two conclusions that both get this wrong
The first is to treat it as closed because the package was pulled in March and the advisory says fixed. The advisory says fixed about the package; it says explicitly that you must treat every credential on an affected system as compromised, and that is the part with no expiry date. The second is to swing the other way and declare open-source AI tooling too risky to use. That reasoning does not survive contact with reality — the alternative stacks have the same shape, commercial vendors have had comparable incidents, and a business that responds by freezing its dependencies simply accumulates unpatched ones. The useful distinction is not open versus commercial. It is whether you know what is in your build environment and how quickly you can rotate everything it can reach.
What should we actually check this week?
This is a short, concrete list, and for most businesses it is an afternoon rather than a project. The point is to produce a written answer somebody owns, not a general intention to look into it.
- Search your dependency history, not just your current lockfile. You are looking for LiteLLM v1.82.7 or v1.82.8 anywhere in build logs, lockfiles, Docker layers or deployment history between 24 March and now — a version you installed once and rolled back still had its forty minutes
- Look for the artefact on disk. LiteLLM's advisory names the file: `litellm_init.pth` in site-packages. If it is there, that machine ran the payload
- Rotate anything the affected pipeline could reach, whether or not you find evidence. Cloud access keys, SSH keys, Kubernetes tokens, database passwords, LLM provider API keys, and any secret sitting in an environment variable or CI configuration. Rotation is cheap; assuming you were fine is not
- Ask your agency and your vendors in writing. A short email — did any of your build systems install these versions, and have you rotated the credentials that touch our systems? — is the only way to see the part of your exposure that lives outside your own estate
- Review what your CI runner can actually reach. Most pipelines hold far broader credentials than any single job needs, usually because permissions were set once during setup and never narrowed
- Pin your build tooling to immutable references. Pinning actions and images to a digest rather than a moving tag is what breaks this specific attack pattern, where a tag is force-pushed to point at new code
- Check whether you would even see it. If a credential of yours were used from an unfamiliar location tomorrow, which alert fires, and who reads it?
Why does AI tooling keep showing up in these stories?
Not because AI code is uniquely insecure. Because of how fast it entered the stack and how little of it went through procurement. Three structural reasons, and none of them are about the technology being bad.
- The ecosystem is young and moving fast. Packages that became infrastructure in eighteen months carry release processes designed for a side project, and the maintainer count has not grown with the install count
- It sits unusually close to secrets. A model gateway's entire job is holding provider API keys and routing traffic, which places it in exactly the part of your system where credentials are concentrated
- It got adopted below the level where anyone reviews things. AI features have been built under deadline pressure across the whole market since 2024, and the dependencies came in with them. This is the same structural gap we wrote about with AI tools arriving through a calendar invite rather than a purchase order — a different door into the same building
The genuinely reassuring part
Everything this incident asks of you is ordinary engineering hygiene that pays off far beyond this one package. Knowing what is in your build environment, holding credentials you can rotate quickly, pinning tooling to digests, and narrowing what a pipeline can reach — that is the same work that limits the next supply-chain incident, whoever it happens to, and there will be a next one. A business that does this in response to a story it was not even directly caught by comes out genuinely ahead. And unlike most security work, there is nothing to buy: this is configuration and discipline, not a product.
Where we fit
The reason this stays open in thousands of companies is not negligence — it is that nobody owns the question. Your developers own the application. Your cloud provider owns the platform. Your agency owns the build they run on their own machines. Nobody owns the join: what is in the dependency tree, what the pipeline can reach, and how fast every credential in it can be replaced. That is the work we do. Retaining JTS for security means somebody senior actually goes and checks — traces the dependency history, finds the pipeline permissions nobody narrowed since setup, rotates what needs rotating without taking your deployments down, and writes the answer down so you can hand it to the customer or insurer who eventually asks. And if it turns out you were caught by this one, the same team runs the response: AI agents on the forensics inside the first hour, a senior engineer making every call, and fixes that are verified, documented and reversible. The businesses that come through incidents like this well are almost never the ones that were luckiest. They are the ones who could answer the question quickly.
Sources
- LiteLLM — Security Update: Suspected Supply Chain Incident (the maintainers' own advisory: affected versions v1.82.7 and v1.82.8, the `litellm_init.pth` artefact, the exfiltration domains, and the instruction to treat all credentials on affected systems as compromised)
- CloudSEK — 2,500+ Companies and 434,000 CI/CD Pipelines Exposed in the Largest AI Supply Chain Breach of 2026 (the research behind the exposure figures, the ~40-minute package window, and the credential categories harvested)
- Trend Micro Research — Your AI Gateway Was a Backdoor: Inside the LiteLLM Supply Chain Compromise (technical analysis of the payload and the Trivy-to-LiteLLM chain)
- SecurityWeek — Over 2,500 Organizations Impacted by LiteLLM Supply Chain Attack (independent reporting on the scale of the incident)
- Cycode — LiteLLM Supply Chain Attack: What Happened and How to Respond (remediation guidance and CI/CD hardening recommendations)
- Tech Times — LiteLLM Supply Chain Hack Hit 2,488 Firms; Stolen Keys Still Work Five Months On (14 August 2026; the reporting that the harvested credentials remain valid, and the FBI advisory on later weaponisation)
