I Built a File Format From Scratch in C/C++

A few days ago, I realized something slightly embarrassing.

I had been working with files my entire life, but I had never really stopped to ask:

What actually is a file?

I knew that a .txt file contains text. I knew that images, videos, executables, and documents all have different formats.

But if I wanted to invent my own file format from scratch, what would I actually have to build?

So I decided to find out.

I built my own binary container format in C/C++.

I called it ZAF.

And along the way, I ended up learning far more about files, bytes, offsets, metadata, directory structures, and binary parsing than I expected.

The idea

The goal was simple:

Take a directory containing arbitrary files and turn the whole thing into a single .zaf file.

For example, suppose I have:

project/
├── hello.txt
├── image.png
└── src/
    └── main.cpp

I wanted to be able to turn that into:

project.zaf

and later give that .zaf file to a reader program and get the original directory back.

That immediately created a much bigger question:

How do you actually store all of this information inside one binary file?

I needed some kind of structure.

Designing the format

I eventually settled on a structure roughly like this:

metadata
    ↓
file records
    ↓
offsets
    ↓
actual file bytes

The beginning of the ZAF file contains information about the files stored inside it.

Each file record contains things such as:

• file name
• path
• size
• offset

The offset tells the reader where the actual bytes belonging to that file begin.

So if I had three files, the format could conceptually look like:

┌──────────────────────────┐
│ Magic bytes              │
├──────────────────────────┤
│ Version                  │
├──────────────────────────┤
│ Directory information    │
├──────────────────────────┤
│ File metadata             │
├──────────────────────────┤
│ File metadata             │
├──────────────────────────┤
│ File metadata             │
├──────────────────────────┤
│ Actual file bytes         │
├──────────────────────────┤
│ Actual file bytes         │
├──────────────────────────┤
│ Actual file bytes         │
└──────────────────────────┘

It sounds straightforward.

It wasn't.

The magic bytes

The first thing the reader needs to know is:

"Is this actually a ZAF file?"

So I gave ZAF a magic-number header.

The reader starts by checking the first six bytes.

If they don't match the expected header, the reader knows that it isn't looking at a valid ZAF file.

This is one of those concepts that sounds incredibly simple until you realize that you are now manually defining exactly what every byte in your format means.

There is no operating system specification telling you what to do.

You are the specification.

Storing directories

The next problem was the directory structure.

A real directory isn't just a list of filenames.

It can contain nested directories:

project/
├── assets/
│   ├── image.png
│   └── logo.png
└── src/
    ├── main.cpp
    └── utils.cpp

So I needed to traverse the directory recursively.

I used C++'s std::filesystem to walk through the directory structure.

The traversal essentially became:

directory
   ↓
find entry
   ↓
is it a directory?
   ├── yes → enter it
   └── no  → record the file

A depth-first traversal worked nicely for this.

For every file I encountered, I collected the information I would eventually need to reconstruct it.

Metadata and offsets

Now came one of the most important parts of the format.

Suppose I have:

a.txt → 100 bytes
b.txt → 250 bytes
c.txt → 80 bytes

If the data for a.txt starts at offset X, then:

a.txt → X
b.txt → X + 100
c.txt → X + 100 + 250

The offset isn't the file's data.

It is simply the position at which the reader should look for that data.

This gives the reader a way to jump directly to the appropriate section of the ZAF file.

Conceptually:

metadata
    |
    | "b.txt is 250 bytes"
    | "b.txt starts at offset 1234"
    ↓
seek to byte 1234
    ↓
read 250 bytes
    ↓
write b.txt

And suddenly I was no longer just "saving files."

I was designing a binary data structure.

Knowing when a section ends

Another problem appeared.

The reader needs to know where the directory information ends and where the next section begins.

I used a dedicated termination marker made from three bytes.

So instead of the reader blindly reading until some arbitrary position, it could essentially do:

read data
    ↓
is this the termination marker?
    ├── no  → continue
    └── yes → section finished

I used the same general idea when parsing the relevant metadata.

It isn't the only way to design a format, but for a small experimental format it made the structure easy to reason about.

Then I had to build the reader

Writing the file was only half the problem.

A format is useless if you can create it but can't read it.

So I wrote a reader that had to reverse the process.

It needed to:

1. Check the magic bytes.
2. Read the version.
3. Parse the directory information.
4. Parse the file metadata.
5. Find the offsets.
6. Jump to the appropriate locations.
7. Read the file bytes.
8. Recreate the original directory structure.

In other words:

.zaf
 ↓
parse
 ↓
metadata
 ↓
offsets
 ↓
file bytes
 ↓
reconstructed files

At this point, I thought I was basically done.

I was very wrong.

Everything was corrupted

I ran the reader.

The directory appeared.

The files appeared.

And then I opened them.

They were corrupted.

This was one of the most useful bugs I encountered during the entire project.

The format itself looked reasonable.

The metadata looked reasonable.

The offsets looked reasonable.

So where was the problem?

I went back through the reader.

Eventually I found it.

The reader was starting the first file three bytes too early.

Those three bytes were part of the structure that came immediately before the actual file data.

So instead of:

[metadata][actual file data]
                  ↑
              start here

the reader was effectively doing:

[metadata][actual file data]
       ↑
   start here

The result was corrupted output.

And this was exactly the kind of bug I had hoped this project would force me to understand.

When you work with high-level abstractions, you can sometimes forget that the computer is ultimately just moving bytes around.

Here, there was nowhere to hide.

Three wrong bytes were enough to break the entire file.

I fixed the offset calculation.

Ran the reader again.

And this time, the files came back correctly.

What building ZAF taught me

The biggest thing I learned wasn't actually how to create a file extension.

It was how much structure exists underneath something as ordinary as a file.

Before this project, words like:

• magic bytes
• offsets
• metadata
• binary layout
• parsing
• file records

felt like disconnected technical vocabulary.

After building ZAF, they became concrete.

I had to decide:

What does byte 0 mean?

What does byte 1 mean?

How does the reader know where one section ends?

How does it find a particular file?

How does it reconstruct a nested directory?

Those aren't abstract questions anymore.

They are design decisions.

The final structure

The final ZAF file ended up following a structure along these lines:

┌───────────────────────────┐
│ Magic bytes               │
├───────────────────────────┤
│ Version                   │
├───────────────────────────┤
│ Directory information     │
├───────────────────────────┤
│ File records              │
├───────────────────────────┤
│ Termination marker        │
├───────────────────────────┤
│ Actual file bytes         │
└───────────────────────────┘

It's certainly not intended to replace established formats like ZIP or TAR.

That wasn't the point.

The point was to understand what it actually takes to design one.

And after several days of debugging, experimenting, and staring at bytes that looked completely meaningless at first...

I had a working file format.

Why I built it

I think this is the part I found most interesting.

You don't always need to build something because it is commercially useful.

Sometimes building something useless is exactly what teaches you how useful things work.

I didn't need another file format.

I wanted to understand one.

So I built one.

And somewhere between calculating offsets, traversing directories, reading raw bytes, and hunting down a three-byte bug, the whole subject became much less abstract.

That's the reason I call myself  Zero Abstraction.

Sometimes the best way to understand what's underneath an abstraction is to remove the abstraction entirely.

The code

The complete project is available on GitHub:

ZAF — source code

If you want to see the entire process as a video, I've also documented the build from beginning to end on YouTube:

Optional: Watch the build on YouTube

The code is the important part, though.

Feel free to inspect it, break it, improve it, or tell me how badly I reinvented the wheel. 😄

Disclosure: I used AI tools as an assistant while working on this article, including for organizing and polishing the writing. The project, implementation, debugging process, and technical experience described here are my own.