Article Summary
- Generative AI has progressed to multimodal assistive tools (real-time captions, voice synthesis, AR), offering major accessibility gains and curb-cut benefits.
- Key risks: bias, inaccuracy (esp. speech/sign language), UI regressions, English-first exclusion, privacy/consent issues, and harm from replacing human support.
- Mitigations: collect diverse consented data, involve disabled people in design/testing, use human+AI workflows, validate technically and experientially, and publish demographic performance with clear governance.
Introduction
Recently, several members of our team attended events on accessibility and inclusive design in Tokyo; specifically the Global Accessibility Awareness Day (GAAD) talks held on May 21st and the Universal Design/ Inclusive Design Talk (ユニバーサルデザイン／インクルーシブデザイン講演会) hosted by HCD-Net, the Human Centered Design Organization on May 26th. Among several illustrious presentations and discussions, one of the hot topics was centered around recent advancements in AI, and its impact in the field of accessibility. For designers in Japan and the U.S., these technologies offer powerful new ways to remove barriers for people with vision, hearing, cognitive, and learning disabilities. They do, however, also risk amplifying existing inequalities when trained or deployed without proper inclusive practices. Let’s take a look at information and real examples shared both at the events and elsewhere to better understand the current landscape of AI development from the point-of-view of inclusive design and better recognize the risks, and best practices to develop these ever-evolving tools for everyone’s benefit.
Evolution: Where AI meets Accessibility
There have been significant leaps in AI’s accessibility development. Early accessibility AI focused on narrow tasks like speech-to-text or screen readers. Now, recent generative models (large multimodal and LLMs) can summarize, translate, generate descriptive audio, create avatars, and synthesize voice and visuals in real time. Some of the industry examples introduced at the events include Google’s Visual Interpreter and TalkBack integrations with Gemini, Meta’s Ray-Ban/Aria device roadmap for on-body assistive AR, and Amazon, Apple, and Microsoft’s recent endeavors into diverse speech datasets. As Masato Horiguchi, a producer at a design firm based in Tokyo, Concent stated in his presentation at the UD/ID Talk, two paths of development are emerging:
Path 1
Retrofitting AI into existing accessibility features
Path 2
Designing accessibility with AI at the core from the start
AI offers a multitude of advantages in accessibility. From real-time, context-aware captions and summaries for people with hearing disabilities to improved visualization and simplification of data and content to aid people with cognitive or learning disabilities, to “personal voice” synthesis tools for people losing their voice to diseases like Lou Gehrig’s disease. Moreover is the potential for a curb cut effect, where improvements designed for disability benefit a broader populations, such as older adults, non-native speakers, or users who are temporarily busy or multi-tasking.
The adoption of AI is accelerating in education, workplace collaboration, and public services, but accessibility development often lags behind mainstream features, creating an AI accessibility paradox where broader deployment increases both potential benefit and disparity.
Key Risks and Failure Modes
1. Bias and inaccuracy
Models reflect the data they were trained on. Studies show higher speech recognition errors for Black speakers and other underrepresented groups (e.g., Koenecke et al., 2020 ). Misclassification of neurodivergent behaviors (e.g., labelling ADHD or autism as “cheating” in some systems) and errors in medical AI due to skewed training cohorts (Warfarin dosing example) illustrate real harm.
2. Accessibility regressions in UI
Large models can produce outputs that violate UI accessibility (contrast, reading order, semantic markup), as observed in some early large-model UIs.
3. Sign language shortcomings
AI sign-language systems are much less mature — avatar-based translations often have <50% accuracy for real-world, technical content and fail to capture grammar, spatial grammar, and cultural nuances of sign languages. Text-driven avatars can be stiff; video-to-video approaches are better but require real signer data.
4. English-first and cultural/linguistic exclusion
Many models and datasets are created off data in English. This risks poor performance in Japanese (and other languages), and mismatched cultural content. One example we heard in the GAAD talks was the markedly worse accuracy for Japanese Sign Language compared to ASL. An English-first model can produce inaccurate subtitles, mistranslated idioms, or sign avatars that assume ASL grammar.
5. Privacy, consent, and representation
Collecting personal voice, sign, and movement data raises consent and ownership issues, especially for vulnerable populations. Without proper governance, datasets can infringe on dignity and cultural sovereignty
6. Over-reliance and replacement risk
AI that replaces human interpreters or support workers can remove nuance, advocacy, and accountability.
Models mirror the data they are trained on. Similar to studies showing higher speech-recognition errors for African American speakers compared to white speakers (Koenecke et al., 2020 study ) or medical errors due to skewed cohorts, underrepresentation of minorities such as those with disabilities can lead to falsifications and real harm. Additionally, large models can cause accessibility regressions in UIs by generating outputs that violate contrast, reading order, or semantic markup. One example given by Yoshihiro Iito, chairman of the NPO for people with disabilities, Information Gap Buster , at the GAAD event was the poor accuracy of AI sign language (less than 50%) compared to subtitles or text-to-speech (~95%).
A final point to mention is the implicit bias towards English in AI. An English-first development approach worsens cultural and linguistic exclusion, producing poor performance in Japanese among other languages, mistranslated idioms, or sign avatars that assume American Sign Language grammar. An over-reliance or over-adherence to uniformity in AI creation and development risks losing nuance, and glossing over individual advocacy and accountability.
The Importance of Including Disabled Persons in Datasets and Design
Representative data and participatory design are essential because statistical learning requires diversity to avoid low accuracy and systemic exclusion. Inclusive datasets and collaborations like the Speech Accessibility Project —adding diverse speech samples (cerebral palsy, dysarthria, low-volume speech)—improve both technical performance and real-world usability, and involving disabled users as co-designers, testers, and data providers from concept through deployment. This raises contextual and experiential accuracy and clarifies where AI should augment rather than replace humans.
Below is a list of design and development practices we at Uism utilize and recommend to mitigate risks:
Inclusive data strategy
- Intentionally collect diverse data: vision, speech (neurodivergent, low-volume, elderly), sign language, and multilingual corpora (other languages).
- Consent-first, privacy-preserving methods.
Participatory, deferred-evaluation processes
- Include people with disabilities at every stage: ideation, design, data provisioning, testing, and governance.
- Use user panels representative of the user in diversity of age, culture, and disability.
Mixed-system design (human + AI)
- Use AI for well-scoped tasks (realtime summaries, auxiliary cues) and keep humans in the loop for contextual or high-stakes work (e.g. medical/legal interpretation, complex sign-language translation).
- Provide easy escalation to human support and transparent confidence indicators for AI outputs.
Multi-tiered validation (technical → contextual → experiential)
- Automate accessibility checks where possible (contrast, semantic labels, reading order), but validate context and experience with real users.
- Measure experiential outcomes: task completion rates, comprehension, cognitive load, and satisfaction across diverse groups.
Localization and sign-language engineering
- Build separate pipelines and datasets for JSL and other sign languages; prefer signer-recorded or video-based avatars for high fidelity.
- Avoid one-size-fits-all avatar grammars; give control to users to select signer style, speed, and verbosity.
Governance, auditing, and transparency
- Publish model performance broken down by demographics and disability type.
- Provide channels for reporting errors and mechanisms to update models rapidly based on real-world feedback.
Final Words
In designing and improving AI capability, or any product/service, it is important to consider the following questions:
- Who, if anyone is being excluded? What tasks must they complete end-to-end?
- How can we diversify the data collected?
- Have we tested the three layers: technical (automated checks), contextual (scenario testing), experiential (real-user validation)?
- How transparent are we in our performance?
- What are our paths for feedback?
Conclusion
Generative AI offers transformative possibilities for accessibility — from scene descriptions for blind users to realtime captions and voice preservation. Yet these gains are fragile: without representative data, participatory design, localization, and accountable deployment, AI can reproduce and amplify exclusion. The mandate for designers is clear: design with (not just for) people with disabilities, invest in diverse datasets, and validate across technical, contextual, and experiential tiers through human testing alongside the AI development. We at Uism are skilled at including the human “voice” in the testing and development of AI. For our collaborative goal of inclusive design early and right, please contact us.
Acknowledgements
We at Uism would like to again express our thanks to the people and organizations who presented and helped orchestrate the recent events in Japan. Your work is invaluable to the betterment of people of all walks of life both here in Japan and around the world. We encourage all of our readers to check out the links to the organizations we have shared above to learn more and it is our mission to continue learning and collaborating alongside these great organizations towards our common mission.
UX Research & Strategy
Learn more about Uism’s UX research services.
