Why you should avoid routing bytes through your Next.js server
Ever lost a 200MB upload at 98%? I have — and it’s the fastest way to lose a user’s trust. Routing large file bytes through your Next.js server (or a serverless function) consumes memory, hits body/time limits, and creates brittle UX on flaky networks.
The production-safe alternative is a three-step pattern that keeps the server out of the data path, makes uploads resumable, and provides real progress to users. This article shows a pragmatic implementation for the Next.js App Router (Route Handlers + client code) using S3 multipart upload and presigned URLs.
Keyword: Next.js S3 multipart upload
The three-step pattern (summary)
- Create an upload intent and presign parts in a Route Handler.
- Upload parts from the browser in controlled, resumable chunks (XHR for progress).
- Finalize the multipart upload server-side and verify the object.
Each step keeps your server memory-safe and makes resume/retry explicit.
Step 1 — create intent and presign parts (Route Handler)
When the client wants to upload a file it sends only metadata (name, size, content-type). The Route Handler validates that metadata (server-side size limits, allowed content-types, optional magic-byte checks), calls S3 CreateMultipartUpload, and returns an uploadId, object key, a chosen part size, and presigned URLs for each part.
Why this matters:
- Your server never accepts the full file body — good for serverless hosting.
- The server enforces policy (size, type, quota) before any bytes are transferred.
- Presigned part URLs let the browser PUT parts directly to S3.
Example: create + presign parts (simplified)
// app/api/uploads/init/route.ts
import { NextResponse } from 'next/server';
import { CreateMultipartUploadCommand, CreateMultipartUploadCommandOutput, S3Client, } from '@aws-sdk/client-s3';
import { getSignedUrl } from '@aws-sdk/s3-request-presigner';
export async function POST(req: Request) {
const body = await req.json();
const { fileName, fileType, fileSize } = body;
// server-side validation
if (fileSize > 5 * 1024 * 1024 * 1024) return NextResponse.json({ error: 'too large' }, { status: 400 });
const s3 = new S3Client({ region: process.env.AWS_REGION });
const key = `uploads/${Date.now()}-${fileName}`;
const create = await s3.send(new CreateMultipartUploadCommand({ Bucket: process.env.S3_BUCKET, Key: key, ContentType: fileType }));
const uploadId = create.UploadId!;
const partSize = 10 * 1024 * 1024; // 10MB default
const parts = Math.ceil(fileSize / partSize);
const presigned = await Promise.all(
Array.from({ length: parts }).map(async (_, i) => {
const partNumber = i + 1;
const url = await getSignedUrl(s3, /* UploadPartCommand */ {
input: { Bucket: process.env.S3_BUCKET, Key: key, UploadId: uploadId, PartNumber: partNumber },
// Note: in real code use UploadPartCommand explicitly
} as any, { expiresIn: 60 * 5 });
return { partNumber, url };
})
);
return NextResponse.json({ uploadId, key, partSize, presigned });
}
(Use the proper UploadPartCommand and typed AWS SDK calls in production.)
Important: configure S3 CORS to expose the ETag header (ExposeHeaders: ["ETag"]) — the browser needs each part's ETag to complete the multipart upload.
Step 2 — upload parts from the browser with controlled concurrency + resume
On the client, slice the File into parts (5–10MB each). Use XMLHttpRequest for each PUT so you get xhr.upload.onprogress events and accurate progress bars. Limit concurrency (3–6 concurrent uploads) so mobile networks and proxies aren’t overwhelmed.
You must record each successful part's PartNumber + ETag. Persist that mapping in IndexedDB or localStorage so a refresh or crash can resume without restarting from zero.
A minimal client example (chunk upload + progress via XHR):
// client/upload.js
async function uploadPart(presignedUrl, chunk, onProgress) {
return await new Promise((resolve, reject) => {
const xhr = new XMLHttpRequest();
xhr.open('PUT', presignedUrl);
xhr.upload.onprogress = (e) => { if (e.lengthComputable) onProgress(e.loaded); };
xhr.onload = () => {
if (xhr.status >= 200 && xhr.status < 300) {
const etag = xhr.getResponseHeader('ETag');
resolve(etag);
} else reject(new Error('upload failed ' + xhr.status));
};
xhr.onerror = reject;
xhr.send(chunk);
});
}
// orchestration (simplified)
async function uploadFile(file, session) {
const { presigned, partSize } = session;
const totalParts = presigned.length;
const completed = {}; // load from IndexedDB/localStorage if resuming
const concurrency = 4;
const queue = Array.from({ length: totalParts }, (_, i) => i + 1);
async function worker() {
while (queue.length) {
const partNumber = queue.shift();
if (completed[partNumber]) continue;
const start = (partNumber - 1) * partSize;
const chunk = file.slice(start, start + partSize);
const presign = presigned[partNumber - 1].url;
const etag = await uploadPart(presign, chunk, (loaded) => {/* update UI */});
completed[partNumber] = etag;
// persist part ETag to IndexedDB/localStorage here
}
}
await Promise.all(Array.from({ length: concurrency }).map(worker));
return completed; // map of PartNumber -> ETag
}
Retry failed parts with exponential backoff. If a presigned URL expires, request a fresh URL for that specific part — no need to restart the entire upload.
Persisting state in IndexedDB allows resumes across reloads. Key the resume entry by file name + size + lastModified (or compute a client-side fingerprint).
Step 3 — finalize & verify
When all parts are uploaded successfully and you have a PartNumber+ETag list, POST that list to a Route Handler that calls CompleteMultipartUpload. As a defensive final step, issue a HEAD on the assembled object to verify size and existence before marking the upload done in your database.
Example finalize Route Handler (concept):
// app/api/uploads/complete/route.ts
import { CompleteMultipartUploadCommand, HeadObjectCommand, S3Client } from '@aws-sdk/client-s3';
export async function POST(req: Request) {
const { key, uploadId, parts } = await req.json();
const s3 = new S3Client({ region: process.env.AWS_REGION });
await s3.send(new CompleteMultipartUploadCommand({
Bucket: process.env.S3_BUCKET,
Key: key,
UploadId: uploadId,
MultipartUpload: { Parts: parts },
}));
// verify
const head = await s3.send(new HeadObjectCommand({ Bucket: process.env.S3_BUCKET, Key: key }));
if (!head.ContentLength) throw new Error('verify failed');
return new Response(JSON.stringify({ ok: true, size: head.ContentLength }), { status: 200 });
}
If CompleteMultipartUpload returns an error, handle idempotency carefully: you can retry completing since CompleteMultipartUpload is idempotent if you send the same parts list. If a session is abandoned, run a periodic cleanup that issues AbortMultipartUpload to free storage.
Production tips and pitfalls
- Use multipart for files > ~100MB. Single PUTs are fragile for very large files.
- Enforce server-side validation before signing (size, content-type, magic bytes for high-risk files).
- Configure S3 CORS to expose ETag via ExposeHeaders: ["ETag"].
- Persist resumable state (IndexedDB preferred) so interrupted uploads resume cleanly.
- Limit client concurrency (3–6) and tune part size (8–16MB typical).
- Remember S3 limits: minimum part size is 5MB (except the last part); max parts 10,000.
- Clean up abandoned multipart sessions with AbortMultipartUpload or lifecycle rules.
Closing thoughts
This three-step Next.js S3 multipart upload pattern gives you resumable, efficient uploads without routing bytes through your server and without hitting serverless time/body limits. It’s not trivial to implement, but it dramatically improves reliability for users uploading large files.
What’s the worst upload failure you’ve seen in production — and how did you recover? Share your story and the lessons you learned.
