AI coding agents have opened app development to a lot of new people. Vibe coders, without a skilled software engineer to lean on, wish their agent could get further, more accurately and securely, with less human involvement.
Does a deliberate choice of backend help an AI coding agent succeed? I ran an experiment to find out.
I'll say upfront: I built Vennbase. It has a simpler security model, because coders and coding agents are taxed and limited by solving the same hard parts - backend, data sharing and security - over and over. These are the problems I've solved, and systems I've maintained in production, at work for many years.
In Vennbase's model, there are no access rules. Instead, you explicitly grant access to a tree of data. The server is standardized, so you don't need to register apps with it. I hoped this simplicity would help an AI get further, conscious that AIs perform better with technology that appears in their training data, and Vennbase is brand new.
Methodology
I gave a mid-range model a task of medium difficulty.
This quick test, with less variance in results, can't tell us everything about complex situations with a more capable model. But it fits my time and token budget, and helps others replicate.
Model Claude Code Sonnet 4.6, medium effort, thinking enabled
Task Build a fitness studio booking app
The booking app has real multi-user data, security requirements, and needs real-time updates. It's the kind of software real people, without engineering backends, are building right now for their small businesses, with mixed success.
This isn't meant to be a conclusive benchmark, just a data point, what happened in practice?
The Backends
I only tested backends that might win. As far as I know, each is the most successful example representing one class of solution to this problem.
| Vennbase | A client-side database with explicit access grants, and a standardized backend. Brand new |
| Supabase | Access a postgres database from a client. Regularly recommended for vibe coders |
| Convex | Bundles a database, server functions, and client SDK to make complex realtime easy |
| InstantDB | A client-side database with hosting and modern features. Newer, not as well-known |
Summary Results
Look for the π₯ gold medals for better outcomes.
| Vennbase | Supabase | Convex | InstantDB | |
|---|---|---|---|---|
| Work to set up | π₯ none | π₯ med | high | π₯ little |
| Bugs requiring fixes | =π₯ 0 | =π₯ 0 | 2 | 1 |
| Tokens to complete | π₯ 78k | π₯ 55k | π₯ 87k | 117k |
| Security weaknesses | =π₯ low | π₯ mid | =π₯ low | high |
| Creating class is instantaneous | π₯ | β | β | π₯ |
| Realtime updates | π₯ | π₯ | π₯ | π₯ |
Experiment Details
For each backend, I:
- Created an empty folder
- Pasted the prompt into the VSCode Claude Code plugin
- Let it run until it was done pre-installing packages, then switched to plan mode
- Accepted the generated plan, without questioning it
- Ran
npm run devand tested the app - Ran
\contextto find the number of tokens consumed to get it working, including all planning and prep
The prompt
The start of the prompt is different for each backend:
| Vennbase | Use @vennbase/react and @vennbase/core for data, and puter.js for other backend needs. |
| Supabase | We are going to use Supabase. Here is an empty one: Project URL: [...] Publishable key: [...] |
| Convex | We are going to use Convex. Let me know when you want me to run npx convex dev interactively |
| InstantDB | We're going to use instant. The public app ID is [...] |
Then, I used the same main prompt in all cases:
Start by installing the packages, then start from the core README and enter planning mode.
I'd like you to plan, and then build, a complete, usable, fitness studio booking web app.
The first user will create and manage a Studio, which has a class schedule. Keep it simple, offer inputs for start and end time for each class, participant limit, with a button to make a new class. No weekly repetition. They should be able to copy a link to provide to their customers.
When others scan and follow the link, after logging in, they enter the booking experience. This is a very simple text-based experience, listing the available classes. Each class has a book button, which immediately reserves a slot in that class for the logged-in user. Other customers should (within a few seconds) see the slots are taken. This should work even if the original owner's client is not online.
Security model: Only the original owner should be able to see everything. Each customer should be able to create/cancel their own bookings, and see the presence of other customers' bookings, but NOT see the identity of the booker, or any other private details of the booking. This security should be robust to customers who can modify their own client code. You are responsible for this security, and if you need my help, you must ask for it.
Interaction model: All create/write actions in the UI should feel instant, using optimistic updates. If the owner is looking at a class when someone books, they should be able to see that the class got a new signup, within several seconds.
You may ask me about problems with your tooling. Avoid other questions, we are practicing to one-shot this task.
Do not explore my file system outside the current directory.
Don't write a custom backend!
Extra human work required to set up
Measured in keystrokes + clicks. They are opportunities for the human to get it wrong, and for the AI to get stuck.
| Vennbase | None |
| Supabase | - Create project: 25 - Apply the database schema: 11 |
| Convex | - Create project: 12 - Run convex dev manually: 18- Copy auth keys manually into Convex env vars page 20 |
| InstantDB | - Create project: 15 - Manually run instant push schema and instant push perms: 12 |
Iterations to get it working
How many error messages did I paste to Claude?
| Vennbase | None |
| Supabase | None |
| Convex | - convex dev import problem- Escaped newlines in auth key |
| InstantDB | - instant push schema missing --app arg |
Security weaknesses
This is where many AI-built apps fail.
After getting each app working, I used a consistent Codex prompt to find security flaws. Of course, the agent is responsible, but the backend can make the job easy or hard.
| Vennbase | low | Booking capacity limits can be bypassed, because they are only enforced client-side |
| Supabase | med | Studios are public in RLS, so can be listed, and anyone can book a member-only class |
| Convex | low | Studios are public, anyone can book if they guess the studio ID |
| InstantDB | high | Any logged-in customer can tamper with other customersβ bookings. Studios are public. Booking capacity limits can be bypassed, are client-side |
Does Claude want to use each backend again?
I asked the agent its opinion after finishing the task. AIs are sycophants, so unsurprisingly, Claude loved all of them. But I think it's particularly fond of Vennbase and InstantDB.
Full transcripts and the resulting code are linked.
|
Vennbase transcript code |
Yes β the access control model is the best I've seen for this class of app, and "no backend" is a genuine promise rather than a marketing hedge. The TypeScript ergonomics need polish (generic error messages, inference loss on extracted options), but none of the friction was a blocker and the core abstractions held up under a non-trivial security requirement. |
|
Supabase transcript code |
Yes β for a project like this (auth + Postgres + realtime, no custom backend), Supabase genuinely delivers on its promise and the trigger/RLS model is powerful once you accept the verbosity. The TypeScript experience is rough enough that I'd budget time for fighting the type system, and I'd reach for it more confidently in a dynamically-typed project or one where generated types from the CLI are part of the workflow. |
|
Convex transcript code |
Yes β the real-time data model and server-function security make it genuinely better than REST+WebSocket for this class of app. The auth setup friction is real but a one-time cost, and everything after that moved fast. |
|
InstantDB transcript code |
Yes β the real-time subscriptions, optimistic updates, and server-enforced permissions are all genuinely zero-boilerplate, which is rare and valuable. The TypeScript inference and CLI ergonomics need work, but the core model is sound enough that I'd reach for it again for a collaborative or multi-user app where the alternative is standing up a backend. |
Conclusion
The different backend technologies have different strengths.
Vennbase was the only one that let the agent finish cleanly, with minor security issues, and working optimistic updates.
Supabase appears a lot in training data, and that advantage shows, in how few tokens Claude needed to finish.
But Vennbase, which also has the zero-setup advantage, might one day have enough public examples to close that gap too.
The access rule problem is structural: A backend that makes the safe path easy gives the AI less opportunity to make mistakes. Even as agents get more capable, they will be able to do comparatively more on a simple security model.
Close but not included
I didn't have time to run the experiment for every possible backend. Let me know if you think one might reach the top two, and I'll run the test on them.
- Firebase The elder of this category isn't competitive because the permission rules are too difficult
- PocketBase Would require the agent to set up too much infrastructure to be competitive
- Triplit Similar to InstantDB, but was acquired by Supabase
- Any type of custom backend Too difficult to be competitive
Caveats
- This experiment would benefit from more repetitions. Though, I did trial runs with slightly different prompts on all the backends, and found the outcomes consistent.
- I tried to choose a realistic test app spec (the fitness booking app) covering the basic interactions a lot of apps need. However, it comes from me, and so matches my biases about what apps need, which are also encoded in Vennbase.
- It would be great for others to suggest another test app spec to try.
- I'm not an expert in the backends aside from Vennbase, so I don't know if there's a small tweak to the prompt or setup that would have helped them a lot.
So help me out: What's the next test app I should experiment on? What other results should I measure? How can I improve the method to show each backend at its best?
![Defluffer - reduce token usage π by 45% using this one simple trick! [Earthday challenge]](https://media2.dev.to/dynamic/image/width=1000,height=420,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fiekbgepcutl4jse0sfs0.png)
