How we open a 4 GB JSON file in a browser tab without running out of memory

Most online JSON tools fall over somewhere between 50 and 500 MB. The reason is simple: JSON.parse (or any parser
that builds a tree) turns every value into a JavaScript object. We measured our own lossless parser at about 13
bytes of heap for every byte of JSON. Chrome gives a tab a heap of about 4 GB, so the ceiling sat near 300 MB.

Here is how JSON Editor Free now opens files of up to 4 GB.

1. Never build the tree

Over 100 MB, the file stays a File. A worker reads it in 8 MB slices (file.slice(a, b).arrayBuffer()) and feeds
them to a resumable byte tokenizer:

It tracks the JSON grammar: objects, arrays, strings, escapes, numbers, literals.
It reports token offsets to a handler, but never creates a value.
It is a state machine, so a token can straddle two chunks.

That one pass proves the file is valid JSON and reports the line and column if it is not.

2. Index only what is big

The handler keeps only containers that span 64 KB or more. For each one it records:

where it starts and ends;
how many children it has;
the byte offset of every 64th child.

It also keeps strings of 64 KB or more. For a 1 GB array of 4.6 million records, that index is about 300 KB.

3. Read rows on demand

Tree view is virtualised, so it only ever asks for "rows 120,400 to 120,480".

The row model knows which containers are expanded and how many rows each contributes, so it can map a row number to "children 120,399 to 120,479 of /data".
It jumps to the nearest checkpoint (child 120,384) and scans forward a few kilobytes.
It skips any big child in one step, using the index.
Each small value is then parsed with the normal lossless parser, so rows look exactly like they do for small files.

4. Search without an index

Search is a single streaming pass over the bytes. A tiny state machine knows whether each byte is inside a string,
a number or between tokens, so a match never spans two values. Only the current match is turned into a JSON
pointer, by walking down from the root with the same checkpoints.

5. Edit a part, not the whole

Editing 4 GB in place is a different project, so huge files are view-only. You can select any value of up to 50 MB
and open it in the other panel as a normal document, with format, sort, transform, compare and save.