Sign Language AI Is Here • What It Should Teach Accessibility Auditors

On August 12, 2026, Google DeepMind announced SL2T, a sign-language-to-text translation model now shipping inside Gboard and Live Transcribe on Pixel 11, starting with American Sign Language to English.

The headline writes itself: "AI can translate sign language now." And honestly, that headline is the least interesting part of the story.

What caught my attention is how they built it. Because the method, not the model, is the part our industry keeps getting wrong.

Key takeaways

• Sign languages are independent natural languages with their own grammar and lexicon, not spoken language performed with hands.
• SL2T is trained on 100,000+ hours across 50+ sign languages, and translates directly from body pose landmarks instead of raw video.
• Google DeepMind built it with Deaf experts, Deaf organisations and a dedicated advisory committee, and published a joint impact report listing the limitations.
• The transferable lesson for accessibility audits: automated output plus expert judgement plus real user involvement. Not one of the three on its own.

What SL2T actually does

Two things are worth understanding before we talk about accessibility practice.

First, this is translation, not transcription. Speech-to-text maps sound to written words inside the same language. Sign language does not work that way. ASL is not English on the hands. It has its own grammar, its own lexicon, and it carries meaning through simultaneous movement of hands, arms, torso, head and face, plus the use of space itself. So the model has to do real machine translation and fine-grained whole-body computer vision at the same time. That is also why sign language gloves, which pop up in the press every few years, were always doomed: they capture the hands and miss the language.

Second, the architecture is quietly interesting for privacy. An on-device model (MediaPipe Holistic) converts the camera feed into pose landmarks, and only those geometric coordinates leave the device. The video is discarded. The team also skipped "glosses", the intermediate word-level annotations most academic work relies on, and translated straight from landmarks to text. Glosses flatten exactly what makes sign languages expressive, so dropping them removes an artificial vocabulary ceiling.

The reported result is a zero-shot score of 70 BLEURT on the FLEURS-ASL benchmark, well above anything published before.

To me, though, the most telling detail in the whole announcement is buried in a caption. Google published the failure cases: rapid fingerspelling misread ("prey" turned into "grey"), dropped classifier constructions, tense lost without context. A benchmark table showing your own errors is not a marketing choice. It is a professional one.

The engineering decisions nobody would have made from a spec sheet

Look at the list of problems the team says it worked on beyond the benchmark:

• streaming latency
• hallucination when the camera sees someone who is not signing
• fairness for the roughly 10% of signers who are left-handed
• one-handed signing, because you are holding the phone with the other hand
• variation across languages, dialects and proficiency levels

I really think this list is the single most useful thing in the article for people who audit interfaces.

Not one of those items falls out of a compliance checklist. Not one of them shows up in an automated scan. They come from watching actual people use the thing in actual conditions. "One-handed signing because the other hand holds the phone" is not a technical insight. It is an observational one, and it's a valuable one.

That distinction is exactly the gap between an interface that passes (compliance) and an interface that works (accessibility).

Building with, not for

The part I keep coming back to: Deaf perspectives shaped every stage. The concept came from a Deaf Googler. Data collection ran through Deaf partners. Evaluation ran through Deaf user studies. And Google set up an AI Sign Language Advisory Committee bringing together global Deaf organisations, then co-signed a joint impact report documenting what the technology can and cannot do today.

Is it also good PR? Sure. Big tech does not publish an impact report out of pure altruism, and a healthy dose of scepticism is always welcome when a platform ships accessibility features tied to a flagship device.

But the pattern is still the right one, and I would rather see it copied than dismissed.

Because the failure mode in our field is so predictable. A team decides what disabled users need, ships it, adds an icon, and moves on. That is the exact logic behind accessibility overlays and their broken promises: a solution designed from assumptions about disability rather than from the lived experience of disabled people, sold as a shortcut around the work.

Involving users is slower. It is also the only thing that reliably tells you whether you built the right thing.

Why this matters for accessibility audits

Here is where it lands for those of us doing audits for a living.

An interface can satisfy every applicable success criterion and still be miserable to use. I have written before about why audits matter more than compliance, and this announcement is a good illustration from the other direction. Google could have shipped a model that scores beautifully on FLEURS-ASL and frustrates every left-handed signer on day one. The benchmark would not have caught it.

Our benchmarks have the same blind spot. A systematic literature mapping on automated tools, expert inspections and user testing makes the point plainly: these three methods surface different types of problems. They overlap far less than people assume. Automated scanning finds a slice. Expert inspection finds a wider and different slice. Users find things neither of the other two will ever produce.

So when a vendor tells you AI will handle your audit, the honest answer is that AI handles a portion of the detection, and detection was never the hard part. Judgement is. Context is. Knowing whether a technically valid alternative is actually usable is.

That is the philosophy behind how I build CheckFox: automate the mechanical parts, then give the auditor room to exercise judgement, document reasoning and collaborate on the parts that need a human. The tool should make the expert faster, not pretend to replace them.

Two things auditors should watch

Sign language is a real WCAG concern, and it is under-tested. WCAG 2.1 and 2.2 include a AAA criterion (1.2.6) requiring sign language interpretation for prerecorded audio content. Most organisations skip it because it is AAA, and I understand the budget reality. But it is worth remembering why it exists: for many Deaf people, written language is a second language, and captions are not a full equivalent. If you have never questioned that assumption in an audit, it is worth revisiting alongside what changed in WCAG 2.2.

Do not let AI features become the new compliance shortcut. Automatic captions have never satisfied WCAG requirements for prerecorded content, and an AI sign language feature on a user's phone does not make your video accessible either. The responsibility stays with the content provider. As European obligations tighten under the European Accessibility Act, expect to see "the user's device can handle it" offered as an argument. It is not one.

Give credit where it is due, and stay critical

To be clear about the scope: SL2T is one direction (sign to text), one language pair (ASL to English), on one device family at launch. Text-to-sign generation is on the roadmap, not in your hands. The Deaf community will judge the real-world quality far better than I can, and the joint impact report exists precisely because the limitations are real.

But the method holds up. Understand the people. Understand their language and their behaviours. Involve them throughout. Then build the technology around that reality, and publish what it still gets wrong.

That is a better standard than most accessibility work is held to, including plenty of accessibility work sold by accessibility companies.

And it is a fair question to bring to your next audit: are you measuring conformance, or are you measuring whether people can actually get things done?

FAQ

What is SL2T?

SL2T (sign-language-to-text) is a Google DeepMind translation model that converts sign language into written text. It powers sign-to-text dictation in Gboard and Live Transcribe, launching with American Sign Language to English on Pixel 11.

Does sign language AI make my website accessible?

No. It is an assistive feature on the user's device for input and conversation. It does not replace captions, transcripts, or sign language interpretation of your own content, and it does not transfer your legal obligations to the user.

Is sign language required by WCAG?

Success criterion 1.2.6 requires sign language interpretation for prerecorded audio content at level AAA. Levels A and AA require captions and audio description instead. Many organisations target AA, but AAA criteria remain relevant when your audience includes a significant number of sign language users.

Can AI replace an accessibility audit?

No. Automated tools, expert inspections and user testing surface different categories of problems, and research shows the overlap is smaller than most teams expect. AI can accelerate detection and reporting. It cannot make the contextual judgements an audit depends on.

Sources:

• Google DeepMind, "Putting sign language AI into users' hands"
• AISLAC joint impact report for SL2T 1.0