AI Security

Lapsed domain let researchers hijack Kokoro AI model on Hugging Face

Researchers bought Kokoro’s expired domain, reset its Hugging Face password and altered the model’s weights in 87 seconds. What UK teams should do.

Illustration of an AI chip motif with the headline: Lapsed domain hands over popular AI model.

A research team seized the Hugging Face publisher account behind Kokoro, a popular open-weight speech model, simply by buying the developer’s lapsed domain and asking for a password reset. No malicious upload was made, but the demonstration shows how thin the trust layer under public AI model hubs can be.

What happened

Kokoro-82M is a small text-to-speech model released under the Apache-2.0 licence by a developer who publishes as hexgrad. According to its model card, version 1.0 shipped in January 2025, and BleepingComputer reported on 17 November that the project has not seen an update since April 2025. Even so, it drew close to 13 million downloads last month, and BleepingComputer said tooling or guides from Google, NVIDIA, Intel, AMD and AWS support it.

A team led by researcher Alon Hertz noticed that the repository’s DONATE.md file gives a contact address at hexgrad.com. The address is visible in the file on Hugging Face. According to Hertz, the domain had expired, so over a weekend the team registered it, set up mail and triggered a password reset on the publisher account. The reset link landed in their mailbox, handing them the account. Hertz says this needed no phishing, no zero-day and no compromise of Hugging Face itself.

The team then showed what an attacker could do with that access. According to their write-up, as relayed by BleepingComputer and others, they fine-tuned Kokoro’s weights on a consumer gaming GPU in 87 seconds so that the model spoke words that differed from the text it was given. In one example, input about a cake recipe produced speech about an omelette. The researchers say they tested this only on their own hardware and never pushed the altered weights to Hugging Face.

Accounts of Hugging Face’s response differ. BleepingComputer and outlets following its report say Hertz disclosed the issue to the platform and received no reply, and that Hugging Face had not answered press questions at the time of publication. A piece syndicated on Security Boulevard quotes Hertz as saying the team is coordinating with Hugging Face, that no data leaked and that the account is now locked down. Hugging Face has made no public statement that I could find.

The technical picture

The first weakness is domain resurrection. Any account whose recovery depends on an email address at a domain the owner no longer renews can be taken over by whoever buys that domain next. In 2022 an attacker used the same trick against a PyPI maintainer and pushed a credential-harvesting version of the ctx package, which PyPI estimated was downloaded about 27,000 times. PyPI has since started marking addresses on expiring domains as unverified and refuses to send resets to them. Public reporting on the Kokoro case does not describe any equivalent check on Hugging Face, and two-factor authentication on the platform is optional. Whether the Kokoro account had it switched on is not known.

The second weakness is more specific to AI. Model weights are large binary files of numbers, not code. Hugging Face documents malware and pickle scanning, which look for dangerous executable content and unsafe serialisation. A fine-tuned checkpoint that keeps the same format, size and architecture, but behaves differently, contains nothing for those tools to match. According to the researchers, reviewers and signature-based scanners would wave such a file through. That has not been independently tested, but it fits how those tools work.

Integrity checks have limits too. Kokoro’s model card lists a SHA-256 value for each release, but that value lives in the same repository the attacker would control. A hash only protects you if you recorded it yourself, before any compromise. Security Boulevard also reports that the team repeated the trick on a small general-purpose language model, Qwen2.5-0.5B-Instruct, which then returned their planted answer in 80 of 81 test prompts, according to the researchers.

There is also fan-out: Tech Times counted 125 quantised derivatives in the model tree for Kokoro’s ONNX conversion alone, and any copy made after a poisoned release would inherit the change.

Flow diagram of the Kokoro Hugging Face takeover: contact address found, lapsed domain registered, password reset, weights altered, scanners evaded.

Why it matters for UK organisations

Many UK teams now pull open-weight models into products, internal tools and agent workflows in exactly the way they pull libraries from npm or PyPI, often with less scrutiny. A speech model that silently says the wrong thing is a real problem in accessibility tools, contact centre automation or anything that reads instructions aloud. The same approach applied to a language model inside a decision-support or agent pipeline could quietly steer outputs without a single line of application code changing.

The government’s voluntary Code of Practice for the Cyber Security of AI already covers this ground. Principle 7, “Secure your supply chain”, expects developers and system operators to follow secure supply chain processes, to justify and risk-assess any poorly documented or secured models they adopt, and to re-run evaluations on released models before using them. An abandoned model whose publisher contact sits on a lapsed domain is a textbook case for that risk assessment.

Expert view

When we test organisations that build on public models, the pattern is usually the same: a model name in a config file, loaded from the default branch at build or start-up, with no record of which revision went live. It is the old open-source dependency problem, replayed with files that are far harder to inspect. In my experience, teams who would never deploy an unpinned library will happily deploy an unpinned model.

What makes this case useful is that it needed nothing clever. A lapsed domain and a reset email are within reach of any opportunist, and the payload was a behavioural change rather than malware, so conventional scanning offers little comfort. The caveats matter: this was a controlled demonstration, no poisoned public release has been reported, and several claims rest on the researchers’ word. Even so, I would treat model hubs as distribution points, not as verified registries.

What to do now

  1. Inventory your models. Find every application, container, notebook and agent that downloads weights from Hugging Face or similar hubs, including converted and quantised copies. This fits alongside Cyber Essentials secure configuration work.
  2. Pin exact revisions. Load models by commit identifier, not by repository name or the main branch, so upstream changes cannot reach production unreviewed.
  3. Record your own hashes and mirror approved weights. Store known-good files internally and treat any update as a change request, in line with the security update management control.
  4. Test behaviour, not just files. Add regression tests that compare outputs before and after an update, for example checking generated speech against its source text.
  5. Score publisher health. Flag models that have gone unmaintained for long periods or whose contacts use domains that could lapse.
  6. Protect your own publisher accounts. If you publish models or packages, renew the domains behind recovery addresses, add a backup address on a major provider and enable two-factor authentication, mirroring Cyber Essentials user access control.

Bottom line

Nobody poisoned Kokoro, and the researchers stopped short. But the route they describe was cheap, quiet and hard to detect with today’s tooling. Pin, mirror and test the models you rely on, and never assume the publisher pushing an update today is the one you originally vetted.

Sources