Over the past week, I have been vibe coding Unity Access, a plugin for the Unity Editor that makes the previously difficult-to-access editor more screen-reader compatible.
Unity Access uses the NVDA API alongside the Unity Editor API to create a bridge between Unity and the NVDA screen reader.
I have been using Codex for the programming, and it is for this reason that I wanted to write this post. During the development of Unity Access, I have learnt a lot about using Codex to develop an accessible application that tutorials and documentation simply do not cover.
One of the biggest lessons I have learnt is that building accessible software with AI requires considerably more than simply telling the model to "make it accessible". Accessibility needs to be expressed as specific, testable behaviour.
In this post, I will describe what worked, what did not work, how you can build your own accessible applications, and some of the things you should watch out for.
Tools and technology
Before I get into the details, I feel it is important to describe the tools and technology I use. This is because some of the results I experienced may differ from those of someone using different tools.
To write the code for this project, I used Codex running GPT-5.6 Sol with medium reasoning effort.
For my work, I found this to provide a good balance between efficiency, cost per one million tokens and quality of output. These factors are particularly important to me because I want to achieve the best possible result while keeping token costs as low as reasonably possible.
To write the Markdown specification files, I used VS Code. I chose this editor because it works extremely well with the NVDA screen reader. This means I can use it out of the box without accessibility modifications.
This is particularly important to me as a blind developer because it gives me a great deal of flexibility and independence. I can navigate the project, read and edit code, organise files and review changes without relying on inaccessible visual interfaces.
To test the plugin, I am using Unity with a sample project based on the Universal Render Pipeline (URP).
I chose Unity because I find its GameObject and component architecture easier to understand and work with. Even though Godot already has an established plugin for editor accessibility, I personally found its tree structure harder to reason about.
Finally, to read what is displayed on my screen, I use the NVDA screen reader.
I use NVDA because it is open source and therefore does not come with licensing fees. In addition, I have found its ability to read my code editor and source code to be superior to JAWS for my particular workflow.
Together, these tools allow me to understand what is happening on my computer and to develop, run and test my code independently.
Building the plugin: what worked and what doesn’t
What worked?
During my experience with different AI models, I have found that each has its own quirks and characteristics. Codex with GPT-5.6 Sol is no exception.
Here is my list of things that worked well, and continue to work well, when developing Unity Access.
- Setting clear boundaries in the root
AGENTS.mdfile.
This might seem obvious, but clear rules and boundaries make an enormous difference.
Tell the model where it is allowed to make changes, which APIs it should use, which files it should or should not modify, how large you want changes to be, what accessibility requirements apply, and what must be true before the task can be considered complete.
The more specific these rules are, the more predictable the model becomes.
This becomes particularly important when developing an accessibility-focused application or plugin.
For example, in my experience, telling the AI:
"The user must be able to navigate this menu using NVDA."
produces much better results than simply saying:
"Make this menu accessible."
The first instruction gives the AI something concrete to work towards.
You can make this even more specific by defining what navigation actually means. For example:
- every interactive control must be reachable using the keyboard;
- NVDA must announce the purpose of each control;
- the current value or state of the control must be available to the user;
- controls must be usable without a mouse; and
- the user must be able to leave or return from the interface using the keyboard.
This gives the AI a measurable target rather than an abstract concept.
I have found this to be one of the biggest differences between simply asking AI to produce "accessible" software and actually getting useful accessibility behaviour.
Setting boundaries around the size of changes is also useful. If you tell the AI that you want a small, direct change, it is less likely to waste tokens modifying or refactoring an entire file when only a small section needs changing.
- Standardising functions such as buttons and text boxes.
Standardising functions for things such as buttons, text boxes and other controls, and placing them in a utilities folder, has been extremely useful.
You can reference these utilities in future Markdown specification files and instruct the AI to call those specific methods. This prevents the AI from recreating the same utility every time it needs one.
More importantly for an accessibility project, it prevents subtle variations from appearing throughout the application.
For example, instead of allowing Codex to invent three different ways of creating an accessible button, I can create one accessible button utility and tell it:
Use the accessible controls located in
./utils.
That makes accessibility behaviour much more consistent throughout the project.
It also means that if I improve one of the shared accessible controls later, every part of the plugin that uses that control can benefit from the improvement.
- Organise your code into related folders.
This is not just useful for the AI. It is also extremely useful for screen-reader users.
Navigating a properly organised folder tree using NVDA is considerably easier than searching through a large collection of files, particularly when files have similar names.
Because I am using a CLI, I can also set the working directory to a particular folder and instruct the AI not to add or modify code outside that directory.
This gives me another layer of protection against the AI modifying working code elsewhere in the project and potentially introducing bugs.
For a Unity project, where the complete project can contain a large number of generated and engine-related files, a properly configured .gitignore is also extremely useful.
This means I only need to commit the actual plugin and relevant project files rather than the entire Unity project.
An organised folder structure also makes relative paths easier to reference in Markdown specification files. Instead of telling the AI vaguely to find a particular file, I can tell it exactly where that file should exist.
- Separating UI functionality from backend functionality in
AGENTS.mdfiles.
Separating UI requirements from backend functionality in my Markdown specification files has worked extremely well.
This is particularly useful when creating an editor plugin.
For example, I can have separate sections describing:
- what the interface should contain;
- how keyboard navigation should work;
- what NVDA should announce;
- which backend functions are required; and
- which Unity Editor APIs should perform those operations.
This means that if I want to change the UI without changing the backend, I can instruct the AI to work only within the relevant section.
Likewise, if a backend function needs replacing, I can make that change without unnecessarily redesigning the interface.
It also makes the specification much easier for me to read and maintain.
If there is a particular function or UI element that I later decide to remove, it is considerably easier to find and remove when the requirements are separated rather than buried inside one enormous paragraph.
- Keeping Markdown specification files concise and structured.
I found that keeping my Markdown files to the point, while adding a short description at the top explaining the purpose of the file, helps keep project requirements clear and reduces the likelihood of the AI misunderstanding what I intend.
Previously, I wrote many of my specifications almost like essays.
That did not work particularly well.
The AI would sometimes overlook tasks or functionality described earlier in the file and concentrate disproportionately on instructions appearing later.
Breaking the specification into properly organised headings and sections does not just make it easier for me to read with NVDA; it also appears to make it easier for the AI to understand which requirements belong together and remain within scope.
The goal is not necessarily to make specifications tiny. The goal is to make every requirement easy to find and difficult to misunderstand.
Now, this all makes it sound as though my project went well. However, here are some things that did not work and that I only found out later.
What doesn’t work
- Having ChatGPT write parts of my Markdown specification files without validating them.
Having ChatGPT write parts of my Markdown specification files initially seemed like a great way to fill gaps in my own Unity knowledge.
However, I discovered that this comes with risks.
Although ChatGPT can research and write code effectively, I found that asking it to define technical requirements for parts of Unity that I did not already understand could cause problems if I accepted the output without checking it.
One example was my project_settings.md file.
Because I did not know exactly what should be included in an accessible Unity Project Settings implementation, I asked ChatGPT to help create the specification.
It not only missed a substantial number of the project settings I actually needed, but also provided Codex with outdated API references.
Codex then implemented those instructions.
The resulting errors caused Unity to enter Safe Mode, and I had to spend additional time and tokens debugging and correcting the implementation.
This does not mean that using AI to help create specification files is a bad idea. I still do it.
The lesson I learnt is:
Do not use one AI's output as unquestioned technical truth for another AI to implement.
If the AI is filling a gap in your own knowledge, that is precisely when validation becomes most important.
Proofread what it gives you, question anything that does not look right and, where possible, compare technical requirements against the current official documentation before allowing another AI agent to build from them.
- Having code for buttons and text boxes scattered around the project.
Before I standardised my buttons, text boxes and other controls, UI-related code was scattered throughout the project.
This did not work particularly well.
If I did not provide Codex with a standard way of creating text boxes, buttons and menus, it would attempt to create its own variations.
Those variations did not always provide the accessibility behaviour I expected.
For accessibility, this is particularly problematic. A user should not have to experience different keyboard behaviour simply because three different interfaces were generated at different stages of development.
I found that it is much better to provide Codex with standardised subsystems that it can reuse rather than expecting it to independently understand your accessibility best practices and reproduce them consistently throughout an entire project.
- Changing my model halfway through the project.
Changing my model halfway through the project turned out to be a mistake.
I switched from GPT-5.6 Sol to GPT-5.6 Lunar because of the potential cost savings.
However, for the particular work I was doing, I noticed a substantial drop in performance.
Lunar was not handling the larger project-level task as successfully as Sol. For my workflow, I found it better suited to smaller, more tightly focused tasks than something as substantial as creating an entire accessible asset viewer.
I ended up using a significant number of additional tokens trying to correct the resulting problems.
Eventually, I returned to GPT-5.6 Sol to repair the implementation. Parts of the asset viewer had to be substantially rewritten, and I also discovered that some of the accessibility requirements I had defined were no longer being followed consistently.
The lesson for me was:
The cheapest model per token is not necessarily the cheapest model for the overall task.
If a less expensive model requires several attempts, introduces bugs or requires another model to repair its work, the apparent saving can disappear very quickly.
Model selection should therefore consider the complexity of the task, not simply the advertised token price.
How to build your own accessibility-focused app or how to modify an existing app to be more accessible
When making an app, if it is intended to be accessibility-focused, it is best to start with accessibility in mind from the beginning.
There is little point building your entire UI and then discovering that screen readers or other accessibility tools cannot interact with it.
You are likely to save both tokens and development time by designing accessibility into the application from the start.
Do your research. Read the latest accessibility guidelines, but also read the documentation for the particular assistive technologies you want your application to support.
If you are using AI agents, provide them with clear accessibility requirements and best practices in the root AGENTS.md file for your project.
Do not simply write:
"Make everything accessible."
Define what accessible actually means for your project.
For example:
"All menus must be usable entirely with the keyboard and every interactive control must expose enough information for an NVDA user to understand its purpose, current state and available action."
Specific requirements give the AI something that can actually be implemented and tested.
Before implementing a custom accessibility solution, also investigate what your framework already provides.
There is little benefit in implementing an entire screen-reader-compatible UI system from scratch if the framework you are using already provides accessible controls that meet your requirements.
Using existing functionality can save considerable development time and allows you to focus on the places where custom accessibility work is actually necessary.
The same principle applies to other accessibility features: use the platform or framework's existing accessibility functionality where it is suitable, and build custom solutions where it is not.
Finally, if you are unsure whether the accessibility features you are implementing actually work for the people you are trying to support, ask them.
Find a forum, Discord community or other group containing the people your application is intended to serve and ask for feedback.
Ask what is confusing.
Ask what their screen reader announces.
Ask what takes too many keystrokes.
Ask what they cannot reach at all.
Most of the time, people will be willing to explain what works, what does not work and what could be improved.
An AI model can help implement accessibility, but it cannot replace feedback from the people who actually depend on those features.
For me, one principle has become particularly clear throughout this project:
Do not tell AI to "make it accessible". Tell it exactly what an accessible user must be able to do.
Contribute to the discussion
Have you ever used AI to make something more accessible?
If so, what worked well for you, and what mistakes or unexpected problems did you encounter?
