Can You Explain the System AI Built for You? — I Audited My Own Web App with 20 Questions

Hello from Japan 🇯🇵

AI has made it possible to build web applications much faster than before.

Paste in an error, and AI can help investigate the cause.

Say, "I want to add a feature like this," and it can write the code.

It can help create a Dockerfile, pytest tests, authentication logic, and more.

It's incredibly useful.

But while continuing to build my own applications, one question started bothering me:

Can I actually explain the system I built?

Code working correctly and understanding why it works are not the same thing.

So this time, I decided to audit my own understanding of the Flask application I'm developing.

The rule was simple:

Answer 20 questions in my own words—without looking at the code, without searching, and without asking AI.

Only after answering would I compare my understanding with the actual implementation.

The results were surprisingly interesting.

This article was originally published in Japanese on Qiita and has been translated and adapted for DEV Community.

The Web Application I Audited

The application is a bakery sales-management system I've been building as a personal project.

Its basic workflow is:

Register products
↓
Enter daily sales quantities
↓
Save sales data
↓
Visualize rankings and charts
↓
Send sales data to the Gemini API
↓
Display suggestions that may help with business decisions

The main technologies include:

• Python / Flask
• PostgreSQL
• SQLAlchemy
• Flask-Login
• Flask-WTF
• Alembic / Flask-Migrate
• Docker
• Gunicorn
• Render
• Gemini API
• pytest
• GitHub Actions

The application works.

At the time of this audit, it had 91 pytest tests.

But this time, I wasn't trying to answer:

"Do I know the names of the technologies I'm using?"

I wanted to answer:

"Can I explain why each one is there and what it actually does?"

I Defined Three Levels of Understanding

I evaluated myself using three levels.

The most important rule was:

Do not look at the code first.

If I looked at the implementation before answering, I could always say:

"It works this way because the code here says so."

But that would not tell me whether I actually understood the system beforehand.

So I first answered entirely from memory.

Only afterward did I inspect the real code.

The 20 Questions

These were the questions I used.

My initial result was:

🟢 6
🟡 10
🔴 4

The purpose was not to compete for a score.

I used it as a diagnostic tool to find:

where my own understanding stopped.

I Could Explain the User-Facing Purpose Relatively Well

One interesting result was that I could explain fairly well:

why I had built the system in the first place.

For example, I understood the role of AI as:

It is not a system where AI makes every decision. It provides reference suggestions based on sales data so that the store manager can think about what to try next.

I could also explain that:

Even if the AI feature becomes unavailable, the core functions—product registration, sales entry, rankings, and charts—still remain usable.

So I had a reasonable understanding of:

the purpose of the system from the user's perspective.

The problems appeared behind the scenes.

I Thought Sales Data Went "Through GitHub" to the Database

One question was:

How does sales data entered in the browser reach PostgreSQL?

My first explanation was something like:

It goes through GitHub and then reaches the database.

That was wrong.

The actual path is:

Browser
↓
Flask application
↓
Input validation
↓
SQLAlchemy
↓
PostgreSQL

GitHub is not part of this runtime data path.

I now separate the roles like this:

GitHub
= Storage for source code and change history

Render
= A place where the web application runs on the internet

PostgreSQL
= A warehouse that stores product and sales data

I knew all three technology names.

But I could not explain them as parts of one coherent system.

That was the gap.

I Also Thought Docker Sent My Finished Local Container Directly to Render

I had another misunderstanding about Docker.

My mental model was roughly:

I build a Docker container on my PC, then send that completed container directly to Render.

But when I went through the actual Dockerfile line by line, my understanding changed.

For example:

FROM python:3.12-slim AS builder

creates a builder stage based on Python 3.12 slim.

Then:

COPY requirements.txt .
RUN pip install --no-cache-dir --user -r requirements.txt

prepares the required Python packages.

After that:

FROM python:3.12-slim

creates another clean stage for production.

In other words, this is a multi-stage build.

I understood it using a Mini 4WD analogy:

builder
= Assembly pit

gcc / libpq-dev
= Tools such as cutters and files needed during assembly

requirements.txt
= Parts list

Production stage
= The actual machine that enters the race

There is no need to load every assembly tool onto the race car.

Only the required finished components are moved into the production stage:

COPY --from=builder /root/.local /root/.local

Then:

COPY . .

copies the application files from the Docker build context into the production image.

I now think of a Dockerfile as:

not the finished cargo itself, but a loading and assembly instruction sheet describing how the cargo should be prepared.

That mental model made much more sense to me.

I Knew the Name SQLAlchemy, but I Couldn't Explain It

SQLAlchemy was one of the biggest discoveries in this exercise.

I had seen the name many times.

It appears throughout my own application.

But when I asked myself:

What is SQLAlchemy?

I couldn't explain it.

Looking at the real code, I found:

from flask_sqlalchemy import SQLAlchemy

db = SQLAlchemy()

and then:

db.init_app(app)

to connect SQLAlchemy with the Flask application.

When saving sales data, the application uses:

db.session.add(sale)
db.session.commit()

My current mental model is:

PostgreSQL
= A warehouse where data is stored

SQLAlchemy
= An intermediary between Python and the database

Alchemy literally means "alchemy," so I decided to remember it as:

a SQL alchemist that helps Python work with database data in a convenient form.

I also started thinking of these operations like this:

db.session.add()
= Put something onto the shipping line as a candidate for storage

commit()
= Finalize this transaction

rollback()
= Cancel the uncommitted changes from this transaction

Strictly speaking, SQL can sometimes be sent to the database before commit().

But the transaction as a whole has not yet been committed.

Before this exercise, I could list SQLAlchemy under "technologies used."

I couldn't explain it this far.

I Was Confusing Cookies with Browser Cache

Authentication exposed another major misunderstanding.

At first, I explained login persistence as:

The browser stores the login information in its cache.

But browser cache and cookies are different things.

Then I inspected the actual implementation and found another mismatch:

the main branch at the time of this audit was not using Google OAuth.

It was using a single-administrator login system.

So even my belief that:

"My application uses Google login."

was out of sync with the implementation I was actually auditing.

I now think of the responsibilities like this:

Cookie
= Entry ticket or wristband held by the browser

Session
= State such as "this user is logged in"

Flask-Login
= Staff who check that state

@login_required
= A gate saying "only authenticated users may continue"

For example:

@login_required
def dashboard():

means that if an unauthenticated user directly visits /dashboard, Flask-Login checks their authentication state.

The implementation at the time of this audit also stored a SHA-256 fingerprint derived from ADMIN_PASSWORD_HASH in the session and compared it with a fingerprint derived from the current ADMIN_PASSWORD_HASH value.

A technical note about Flask sessions

Flask's default session is not necessarily a separate "box" stored on the server.

By default, Flask stores session data in a signed—not encrypted—cookie on the browser side.

For the purpose of understanding responsibilities, I separate the concepts like this:

Cookie = entry ticket

Session = entry state

This was another area where:

"It works" had made me think I understood it.

I Was Mixing Up APIs and API Keys

When I asked myself what an API was, my first answer was:

An API is something like a key.

That was another misunderstanding.

Now I separate them like this:

API
= A defined interface through which different applications or services communicate

API key
= Authentication information used to access that interface

Environment variable
= A way to keep values such as API keys outside the source code

The application retrieves the Gemini API key using:

api_key = os.environ.get("GEMINI_API_KEY")

and prepares the Gemini client with:

client = genai.Client(api_key=api_key)

So this does not mean:

Use the API key to log into the operating system.

os.environ.get() means:

Retrieve the value named GEMINI_API_KEY from the environment variables available to the running application.

.gitignore Is Not a Feature That Hides Secrets

I also had an inaccurate understanding of .gitignore.

I thought of it as:

Something that hides variables people should not see.

That's not quite right.

.gitignore is:

a set of rules telling Git which untracked files and directories it should ignore.

The .gitignore I inspected during this audit included entries such as:

過去売上高/
*.xlsx
*.db
.env

Using a logistics analogy:

It is a "do not load" list for the truck heading to GitHub.

.gitignore itself does not encrypt anything.

And if secret information has already been committed to Git, adding the file to .gitignore afterward does not remove that secret from the existing Git history.

Trying to explain it forced me to understand that distinction much more clearly.

What Am I Actually Sending to Gemini?

I also inspected the AI part of the application.

Initially, I thought:

I'm sending the prompt written in the dashboard together with the sales numbers to Gemini.

In reality, the Gemini prompt-generation logic is separated into prompts.py.

For example:

def build_sales_prompt(sales_summary: str) -> str:

Then app.py creates a sales summary:

sales_summary = ", ".join(
    [f"{name}: {qty}個" for name, qty in ranked_sales]
)

prompt = build_sales_prompt(sales_summary)

This inserts product names and sales quantities into the prompt.

For example:

Croissant: 80 units
White bread: 65 units
Melon bread: 42 units

Then the request is sent to Gemini:

response = client.models.generate_content(
    model=config.GEMINI_MODEL,
    contents=prompt,
)

and the returned text is displayed on the dashboard.

So the flow is:

PostgreSQL
↓
Aggregate sales data in Flask
↓
Product names + sales quantities
↓
Build instructions in prompts.py
↓
Gemini API
↓
Receive suggestions
↓
Display them on the dashboard

Saying "It Analyzes Current Social Media Trends" Was an Overstatement

This was one of the biggest discoveries in the audit.

The prompt I inspected at the time asked Gemini to consider things such as:

Recent food trends
Potential social media appeal
Patterns involving weekdays, seasons, and holidays

Because of that wording, I had started thinking:

Gemini is actively looking up current social media and internet trends and using them in its analysis.

But when I inspected the implementation, the store-specific information sent to Gemini by this feature was basically:

product names + aggregated sales quantities

There was also no implementation in this flow that retrieved current web-search results or social media data and passed them to Gemini.

So:

Writing "consider recent trends" in a prompt

and:

actually searching the current web and analyzing fresh information

are different things.

I noticed another related issue.

The prompt also asked Gemini to consider:

latent demand based on weekday, seasonal, and holiday patterns

But the store data sent to Gemini was primarily product names and aggregated sales quantities.

The application was not directly sending detailed daily or weekday-level sales records to Gemini in this flow.

So:

Asking the AI to analyze weekdays

and:

giving the AI enough data to perform weekday analysis

are also different things.

The lesson was:

Writing "search for it" in a prompt
≠
Implementing search

Writing "analyze weekdays" in a prompt
≠
Providing enough weekday-level data for that analysis

This was probably the clearest example in the entire audit of:

a difference between what I thought my working application could do and what the implementation actually did.

If I genuinely want future recommendations to include current web information or detailed weekday trends, I would need additional design work such as:

• Explicitly integrating a web-search capability
• Sending daily or weekday-level sales data
• Defining the analysis period and conditions clearly

Even 91 GREEN pytest Tests Do Not Mean "Safe"

One area I could explain relatively well was pytest.

At the time of this audit, the application had 91 pytest tests.

Why had I grown the suite that far?

Because I assumed:

Users will not always operate the system exactly the way I expect them to.

Examples include:

• Sending a product ID that does not exist
• Registering sales against a product from another month
• Registering sales for a discontinued product
• Directly accessing a protected URL without authentication
• Tampering with a CSRF token
• The Gemini API returning an error
• A database operation failing halfway through

In truck driving, I cannot think about safety only in terms of:

"Normally, nobody would do that."

Unexpected things happen.

So I try to think about dangerous scenarios before they occur.

I approach pytest in a similar way.

I have been growing it into:

an "incident prevention log" that records failure patterns before they can become repeat incidents.

But:

91 passed

does not mean:

There are no bugs.

It means:

All 91 expectations represented by the tests at that point passed in that run.

If I discover a new blind spot, I fix it and add another regression test.

I used to feel reassured simply by seeing GREEN.

Now I try to ask:

"What exactly does this GREEN result guarantee?"

Not Being Able to Explain Something Wasn't Embarrassing — It Showed Me What to Learn Next

At first, I felt a little embarrassed.

This was my own application, yet I discovered that:

• I couldn't explain SQLAlchemy
• I was confusing cookies with cache
• I was confusing APIs with API keys
• I thought GitHub was part of the sales-data path
• I thought my local Docker container was simply sent to Render
• I thought the application was retrieving current social media trends

But my perspective changed during the exercise.

Finding something I don't understand means I've found the next place to learn.

What seems more dangerous is:

continuing to use something while only thinking that I understand it.

Being Able to Explain Something Simply Does Not Mean Complete Understanding

There is an important limitation here.

Being able to say:

PostgreSQL is a warehouse.

does not mean I completely understand PostgreSQL.

Calling SQLAlchemy an intermediary does not mean I understand all of its internals.

What I did here was:

use explanation as a way to locate gaps in my understanding.

Being able to explain something is not the final goal.

I see it as:

one diagnostic method for checking understanding.

I Don't Need to Avoid Technical Terms

I also don't think technical terms need to be completely removed when explaining a system.

What matters is:

Can I translate them into words the listener can understand?

For example:

PostgreSQL
= A warehouse that stores application data

SQLAlchemy
= An intermediary between Python and the database

Dockerfile
= A loading and assembly instruction sheet for building the application

GitHub
= Storage for source code and change history

Render
= Rented infrastructure where the application runs

Cookie
= Entry ticket

Session
= Entry state

Flask-Login
= Entry-management staff

API
= A communication interface with another service

API key
= Authentication information used to access that interface

Combining:

the official technical term + language I personally understand

made the system much easier to organize in my head.

Writing Your Own Manual Can Be a Surprisingly Good Test

One thing I learned from this exercise is that asking:

Can I write the manual for my own system?

is a surprisingly powerful test.

For a user-facing manual, I need to explain:

What can the application do?
How do you operate it?
When is it useful?
What can it not do?

For a developer-facing explanation, I need to explain:

How does the data flow?
Why is this technology being used?
How does authentication work?
What is sent to the AI?
What remains available during failures?
What do the tests actually guarantee?

If I get stuck while trying to write one of these explanations, that may point to a gap in my understanding.

And if I find myself thinking:

"This operation is extremely difficult to explain."

the problem may not only be my understanding.

There may also be a UI/UX problem.

Writing documentation can itself become a system inspection.

This Is Not an Argument Against Using AI

This article is not saying:

Using AI to write code is bad.

I will continue using AI.

Without AI, I probably could not have experimented this much in such a short period of time.

But instead of stopping at:

AI created it
↓
It works
↓
Done

I think there is more value in continuing:

AI created it
↓
It works
↓
Why does it work?
↓
What does this component do?
↓
If I break it, will pytest detect it?
↓
Can I explain it to the user?
↓
Does my understanding match the actual code?

At that point, AI stops being only a code-generation machine.

It becomes a powerful learning tool.

Summary

I audited my own web application using 20 questions.

My initial result was:

🟢 6
🟡 10
🔴 4

But the score itself was not important.

Comparing my answers with the actual code revealed gaps such as:

• GitHub is not part of the runtime path for sales data
• A Dockerfile is an assembly instruction, not the finished product
• SQLAlchemy helps connect Python application logic with the database
• Cookies, sessions, and Flask-Login have different responsibilities
• An API and an API key are different things
• .gitignore is not an encryption feature
• I verified what data was actually being sent to Gemini
• My belief that the application was analyzing current social media trends did not match the implementation
• What I ask an AI to analyze is not necessarily the same as the data I actually provide for that analysis

The biggest lesson was:

What I thought I knew was more dangerous than what I knew I didn't know.

In the AI era, the speed at which we can generate code will probably continue to increase.

That may make it even more important to spend time asking, in our own words:

What is this here for?

How does it work?

What happens if it breaks?

Can I explain it to the user?

I want to continue building with one rule in mind:

Don't stop at "it works." Keep going until I can explain it.

pytest Improvement Series

• Stage 1 — From 3 Simple Tests to 9

• Stage 2 — Strengthening Sales and Product Registration

• Stage 3 — Authentication, CSRF, and Access Control

• Stage 4 — Testing Failure Scenarios

• Stage 5 — Questioning GREEN with Mutation Testing