Home » Disability representation in AI: Lost somewhere in the training data

Disability representation in AI: Lost somewhere in the training data

Despite AI’s rapid evolution, disability remains a blind spot in both its data and design thinking.

by Anagha BP

Disability representation in AI is a lot more than just about fairness. It is about whether technology can reflect reality, or choose to erase it. When the most advanced tools fail to include everyone, the future we are building becomes dangerously one-sided.

A few months ago, the internet couldn’t get enough of those AI-generated Ghibli-style portraits. While it looked like harmless fun, we at ChangeinContent had already questioned how such trends crossed into the exploitation of art and the overlooked pressure AI puts on the planet. Moreover, it also reminded us that even in the most creative spaces, inclusion is not guaranteed for everyone.

No doubt AI is changing how we live, making tasks faster and access easier. But at the same time, many people are being left behind, especially those with disabilities. Repeatedly, AI has tried to hide or ‘fix’ a disability.

AI’s blind spot: Disability representation in AI is still missing

In many cases, AI still lacks the ability to accurately represent people with disabilities. Even within the viral Ghibli trend, users with disabilities requested portraits of themselves, only to receive results that altered their appearance entirely. A person who uses a wheelchair was shown standing. Some were given prosthetic arms or legs, but they never mentioned it.

Australian Paralympic swimmer and disability accessibility consultant Jessica Smith shared her own experience with AI in a TIME article. Born without her left arm, she attempted to generate images of herself using AI tools. Despite clear instructions to reflect her real appearance, the tools gave her two arms or replaced the missing one with metallic or prosthetic substitutes she never asked for. Even after multiple prompts to represent her disability authentically, the AI failed every time.

From inclusion to erasure: How AI ‘Fixes’ what is not broken

I tried a small test myself. I asked ChatGPT to describe a group of ten friends. It featured a diverse mix of men, women, and non-binary individuals across various skin tones. But no disabled characters. I thought maybe the number was too small, so I asked for 20. Still none. Only after I directly brought up the lack of representation did the system change its answer and include two visibly disabled characters and one with an invisible disability.

AI Prompt to check Disability Representation in AI

That representation was not automatic. It had to be requested.

Bias, caution, and the silent exclusion of disability

It’s not hard to figure out why this keeps happening. AI systems learn from the data they are trained on. If the data mostly shows able-bodied people or treats disability as something to be corrected, then that’s what the system will repeat.

The funny thing is, when you ask AI why it lacks disability representation, it says yes, there is bias in training, and there is also a fear of getting it wrong. It said AI, just like humans, avoids it because they are worried they might misrepresent the experience or fall into stereotypes. So, why not work with people with disabilities when training AI models? Talk to them, learn from their actual experiences, and use that to train AI properly. You know, like one would when trying to accurately represent any group.

Instead, the current excuse is fear. The fear of getting it wrong. But AI is still getting it wrong. And worse, that fear has led to just erasing disabled people altogether. In trying so hard not to misrepresent them, AI chose not to represent them at all.

When AI learns from Ableism, it gets disability wrong

AI is learning from a world where ableism is everywhere in data, language, and media. The 2024 Access Survey examined how individuals with disabilities feel about their portrayal in the media. 57% said they do not see fair representation in TV, advertising, film, or books. 73% felt that public attitudes toward disability have either remained the same or worsened. Moreover, 97% of the top one million websites fail to meet basic Web Content Accessibility Guidelines (WCAG), which are the global standard for accessible design. So when AI trains on this digital content, it learns from material that already excludes disabled users.

As a result, even when AI attempts to generate images that include accessibility tools, it often gets them wrong. For example, AI-generated images have shown hearing aids that don’t resemble any real-world models despite claiming to portray realistic visuals. White canes, used by blind or low-vision individuals to detect obstacles, are often depicted as being dragged behind a person when, in reality, they are meant to sweep in front to guide each step.

These mistakes are the result of a system that never took the time to understand disability.

Disability representation in AI: Asking better questions

AI is now part of almost everything we do, so it becomes even more important to ask who is being included in these systems, and who continues to be left out? If we want AI tools that truly work for everyone, people with disabilities must be part of the process from the outset. The work begins by asking better questions. Who is in the training data, who is testing the product, and who is sitting at the table when decisions are made? And most importantly, who is still missing? Fixing the system takes time, but if we don’t fix the data it’s learning from, we’re just teaching AI the same mistakes all over again.

At ChangeinContent, we believe inclusion in technology cannot be selective. If AI is building the future, then let it be trained on humanity in all its forms, not just the most visible ones.

Disclaimer: The views expressed in this article are based on the writer’s insights, supported by data and resources available both online and offline, as applicable. Changeincontent.com is committed to promoting inclusivity across all forms of content. We broadly define inclusivity as media, policies, law, and history, encompassing all elements that influence the lives of women and marginalised individuals. Our goal is to promote understanding and advocate for comprehensive inclusivity.

Leave a Comment

You may also like