I measured the speed and monthly cost of Amazon S3 Files, which lets you use S3 as files (and how it differs from FSx for ONTAP)

Conclusion first: the one-client speed limit, scaling with client count, and a monthly cost order that splits by object size

Amazon S3 Files lets you open, read, and write the data in S3 like an ordinary folder. With plain S3, a program has to call S3-specific APIs. With S3 Files, tools built to open files can work on your S3 data as they are.

This post is a record of measurements and estimates for people who do not run storage performance tests as a routine job, to help decide whether S3 Files belongs in a design. I built the environment myself in ap-northeast-1 and measured speed on 2026-10-10. The monthly cost is not a measurement. It is arithmetic from list prices read on 2026-08-09. When I re-read the 2026-08-09 record with the Price List API on 2026-10-16, the values were the same (it covers 10 items and does not include the two PUT request prices).

Three things came out of it.

• One server has a read speed limit. On a c5n.9xlarge (a 36 vCPU server) it was about 1.1 GB per second (1,104.9 MB/s). It stops when the relay program running on the server (efs-proxy) uses up one CPU core (96.8 to 97.9%). One run.
• Adding servers raises the total. Going from 1 to 8 c5n.2xlarge servers raised the total from 784.4 MB/s to 6,282.6 MB/s (8.01x). All servers read the same 600 GiB file.
• Under this model's base assumptions, the monthly cost order against FSx for ONTAP splits by object size. Amazon FSx for NetApp ONTAP (FSx for ONTAP below) is an AWS managed service that provides NFS and SMB shared folders, and with S3 Access Points attached it also serves the same data through the S3 API. I fixed the stored volume at 20 TiB and estimated five patterns that differ in monthly read and write volume (P1, P4, P5, P6, P8; the contents are in the table in the chapter "Monthly cost estimate"). At 64 KiB, FSx for ONTAP was cheaper in all five ($1,783.11 to $2,218.13 a month, against $7,928.32 to $12,030.81 via S3 Files). At 4 MiB, S3 Files was cheaper in all five ($512.14 to $543.52 a month, against $1,781.28 to $1,788.08 on FSx for ONTAP).

The monthly amounts are subtotals of the main line items for ap-northeast-1, with prices read on 2026-08-09. A main-line-item subtotal is the sum of storage, S3 Files high-performance storage (a low-latency layer inside the file system; you pay storage charges for what it holds), fixed bandwidth, requests, and so on, and it leaves out any item that is not in the tables. They are price x quantity arithmetic, not measurements, and they are not invoices. The FSx for ONTAP capacity pool (the tier that holds data that does not fit on SSD) read request count is an assumed value that I have not confirmed. The P5 write volume and the read and write volumes of P1, P6, and P8 are placeholders with no source.

Because the order was the same in all five patterns, what splits the order in this model is the object size, not the read and write volume. The candidate for the dividing line is the S3 Files import threshold (the file-size boundary for importing into high-performance storage; default 128 KiB). I estimated only two sizes, 64 KiB and 4 MiB, and I did not estimate the sizes at the boundary.

The takeaways: when one server is not fast enough, add servers (confirmed up to 8). Under the base assumptions, FSx for ONTAP was cheaper at 64 KiB and S3 Files was cheaper at 4 MiB. The order can flip when the assumptions change, so replace them with your own values and check. The assumptions to replace are the share of data on the capacity pool, the storage efficiency of the SSD tier, the active share of S3 Files high-performance storage, the S3 Files import threshold, and the FSx for ONTAP deployment. The table of values at which the order flips is in the section "Baseline assumptions and sensitivity" of the generated document in the repository (the link is in the "Assumptions under which the order can flip" section of the chapter "Monthly cost estimate"). This post gives only the directions. The estimate itself is in the chapter "Monthly cost estimate".

How to read this post by goal

• You only want the conclusion: this chapter is enough.
• You want to know whether it works in your setup: "The value of S3 Files and four prerequisites"
• The mount does not proceed: "A hung mount"
• You want to know how it differs from FSx for ONTAP: "Comparing two ways to use the same data through two entry points"
• You want the cost: "Monthly cost estimate"
• You want to know which setup to consider first: "Choosing between S3 bucket first and file system first" at the end of the "Monthly cost estimate" chapter, and "Closing"
• You want the measurement conditions and limits: "The single-mount ceiling", "An aggregate that scales with client count", "An invalid warm / cold comparison", "Status of the four stated values", "What I have not measured"
• You want to reproduce it: "Reproducing it yourself"

Introduction

There are times when you want to handle S3 data as files: an existing script that opens the files in a folder one by one, or a business tool built around a shared folder. Amazon S3 Files became generally available in April 2026, and it lets you mount an S3 bucket as an NFS file system (the network shared-folder mechanism of Linux) without migrating data.

Before using it, I think you wonder about three things.

• What is it good for, in the end?
• How fast is it, and what does it cost?
• How does it differ from a file-server type of service (FSx for ONTAP)?

This post answers the second and third. The speeds are values I measured in my own environment, and the monthly cost is a unit price x quantity estimate. For the differences from FSx for ONTAP, I put my past measurement records (the environment differs per measurement) and the official documentation side by side in a table. I do not cover the first, the introduction. The Classmethod article (Japanese) compares I/O performance and sync lag with Amazon EFS, and the Serverworks article (Japanese) sorts out five services, including Mountpoint for Amazon S3.

In the previous post (S3 Burst Part 2: "The throughput ceilings of three paths") I compared three paths, S3 Files among them, side by side and asked where each one tops out. This time I narrow down to S3 Files alone. Last time the question was which path tops out where. This time it is how S3 Files behaves as a file system.

The mount failed before I could measure anything, so I recorded the cause first. Someone else can hit the same wall. After reading, you can decide, on numbers that come with their environments, whether S3 Files is a candidate, how far the number of servers can carry your speed, and which of S3 Files and FSx for ONTAP with S3 Access Points (the S3 API entry point that you attach to a volume) fits better.

What I cover / do not cover

Covered

• The prerequisites of S3 Files and the value AWS states for it. Prerequisites decide first whether you can use it at all.
• Why the mount failed. It was missing permissions, not the network, and knowing in advance avoids it.
• The single-mount ceiling (c5n.9xlarge). The per-client ceiling is the starting point of capacity planning.
• Aggregate growth with client count (c5n.2xlarge, 1 to 8 clients). It shows how far clients can carry you past one mount's limit.
• The comparison with FSx for ONTAP S3 Access Points. Both give you the same data through files and the S3 API, but the source of truth, the speed of propagation, the growth with client count, and the billing items differ.
• The monthly cost estimate. With the stored volume fixed at 20 TiB, five read and write patterns, and two object sizes, I line up the main-line-item subtotals of S3 Files and FSx for ONTAP. It separates this model's result, in which the size splits the order, from the assumptions under which the order can flip.
• The accounting assumption for S3 Files metadata charges, and the reconciliation of usage against the bill. It separates what the reconciliation showed from the fact that the invoiced amount itself is unconfirmed.
• Reading data on the capacity pool. It separates the existence of billing items from how far a latency observation can be read.
• How my warm / cold comparison failed. Warm means reading data that sits in S3 Files' high-performance storage; cold means reading straight from the bucket. Someone repeating the measurement can fall into the same trap.
• Which of the four values AWS states I could confirm, and which I could not. It separates what is safe to base an adoption on.
• What I have not measured. It lets you judge where the numbers apply.
• How to reproduce it in your environment. It points to the runbooks so you can follow the measurements.

Not covered

• An introduction to S3 Files, and how it differs from Mountpoint for Amazon S3 and Amazon EFS. Articles already cover that, and it is outside this post's subject, measured speed and monthly cost.
• Which configuration is faster. The paths were measured in different environments (NFS version, instance type, configuration), so no same-conditions winner exists.
• Measurements that separate the propagation conditions. The window of about 60 seconds between writing a file and reading it through the S3 API appears as two rows in the comparison table of "Comparing two ways to use the same data through two entry points", and I explain it in four stages. Measurements that separate conditions, such as long continuous appends and concurrent updates, exist only as a plan and have no measured results. The conditions of that window are in another verification record (Japanese). It is outside this post's subject, which is throughput and client count, and that measurement covers a single condition.
• An explanation of the values at which the order flips. The table of values is in the generated document, and this post gives only the directions, because the values have to be read with the sourced ranges and the mechanical search ranges kept apart. I did not estimate the boundary object size (around 128 KiB).
• Invoiced amounts. The monthly cost is a unit price x quantity estimate, and I have not confirmed that it matches a bill.
• For the warm / cold difference, which path a first read takes (straight from the bucket, or from high-performance storage after import). I re-measured the latency with a different design, but DataReadBytes was the same for the first and second reads, so I cannot distinguish the path. The values I measured are in the chapter "An invalid warm / cold comparison" (verification record).
• The tiering difference from a client outside AWS, and from a standby-AZ client. I measured Single-AZ and the preferred AZ of Multi-AZ first generation from an EC2 instance inside a VPC (verification record). The chapter "What I have not measured" lists the remaining scope in a table.

Terms used in this post

Each term is explained at first use in the text. This table is an index for rereading.

The value of S3 Files and four prerequisites

S3 Files lets Linux compute mount a bucket as an NFS file system and read and write it while the bucket stays the source of truth. It is built on Amazon EFS and became generally available in 34 regions in April 2026 (What's New). Four prerequisites decide first whether you can use it.

Four prerequisites

How the mount works

When you mount, efs-proxy (the local proxy mentioned in the summary) starts on the client's EC2 instance. Your application mounts 127.0.0.1, and efs-proxy connects to the mount target in the VPC. The mount target is the NFS endpoint of the file system (port 2049), with a limit of one per AZ. In the verification records, this efs-proxy handles TLS termination and NFS translation in one process (verification record, Japanese). This structure sets the single-mount ceiling measured in a later chapter.

Reads take one of two paths. Small files (under 128 KiB by default) are imported asynchronously into high-performance storage, the low-latency layer inside the file system, on first access, and later reads are fast. Reads of 1 MiB or more go straight to the bucket even when the data sits in high-performance storage. The source is Performance specifications.

Values AWS states

The following four points are AWS's statements. I could get numbers for the second point, propagation time between files and the S3 API, and for the fourth point, single-mount throughput and aggregate growth with client count (up to 8). I did not measure the first, and the comparison for the third was invalid.

• Existing file-based tools work as they are. The product page says "No code changes, no custom connectors, and no new APIs to learn" (product page).
• The same data is usable from both files and the S3 API at the same time. The product page's wording is "nothing to reconcile between two separate systems", meaning you do not have to reconcile two systems.
• Only frequently read data sits in high-performance storage, and you pay for what it holds. Reads of 1 MiB or more cost only the standard S3 GET price (product page).
• Many clients can use it at once. The limit is 25,000 connections per file system, and the aggregate read throughput is "up to terabytes per second" in the AWS pages' wording (quotas, Performance specifications).

If your requirements include NFSv3 or fixed SMB devices or Windows workflows, S3 Files drops out at the prerequisites table. The AWS documentation does not mention mounting from outside AWS that I could find. If you need it, check in your own environment before you put S3 Files on the candidate list. If none of those apply, read the following chapters on the mount failure, the single-mount ceiling, and client scaling, and use them for capacity planning.

A hung mount: missing CloudWatch permissions on the client's IAM role

mount -t s3files timed out every 15 seconds. The cause was not the network or DNS. The IAM role on the client EC2 instance lacked logs:CreateLogGroup and cloudwatch:PutMetricData. Adding the permissions let the mount complete.

The environment was a c5n.9xlarge (Amazon Linux 2023, amazon-efs-utils 3.3.2) in a dedicated measurement VPC in ap-northeast-1 (a public subnet with an internet gateway). It happened while I prepared the measurement, and my troubleshooting notes carry no date. The throughput measurements in the same environment ran on 2026-10-10.

Symptom: timeouts every 15 seconds with a healthy network

• efs-proxy: running, with a TCP connection to port 2049 on the mount target (ESTAB in ss)
• Reachability: both S3 on 443 and the service endpoint on 2049 were reachable
• Mount: mount.nfs4 127.0.0.1:/ still did not finish

Cause: efs-proxy restarting over and over because log creation failed

The efs-proxy log contained these lines.

WARN  cw_publisher Failed to create log group: AccessDeniedException ... logs:CreateLogGroup
ERROR nfs::nfs_reader Error sending message batch to dispatcher DispatchingFailure
INFO  controller Proxy incarnation restarted, restart_count=3

Before establishing the NFS connection, the mount helper tries to create a CloudWatch Logs log group (/aws/efs/utils). When that call fails for lack of permissions, efs-proxy keeps restarting and the mount never finishes. An AWS article names the same behavior as the root cause when a mount hangs forever without an obvious reason (Mounting S3 Files on EC2 in private subnets).

My subnet has an internet gateway, so CloudWatch Logs was reachable. Permissions were the blocker, not reachability. In a private subnet with no CloudWatch Logs VPC endpoint, unreachability produces the same symptom.

Original record: Mount pitfalls (causes and fixes) (Japanese)

Fixes: any of three options works

I did not pick option 2 because the next chapter's measurement uses efs-proxy's CloudWatch metrics (connectivity and byte counts) to isolate the cause of the ceiling.

Pre-mount checks: three items

• IAM role: it carries the permissions from fix 1.
• botocore: installed with dnf install python3-botocore (pip3 does not work). Without it, the mount either stopped with access denied by server while mounting, or went through but CloudWatch metrics were unavailable. The two verification records show different symptoms, and neither records the condition that decides which one you get.
• Mount target: it is in the same AZ as the client, and the security group allows 2049 from the client.

Check these three items before you mount, and decide beforehand whether to grant the CloudWatch permissions or disable logging.

The single-mount ceiling: one efs-proxy core saturated

A single mount stopped at 1,104.9 MB/s for 1 MiB sequential reads (c5n.9xlarge, one run). The limiter was the efs-proxy process's CPU: one core of 36 vCPU was full (96.8 to 97.9%). As noted in the prerequisites chapter, S3 Files does not support nconnect, so no mount option can add connections to efs-proxy and move this ceiling.

Measurement conditions

• Environment: ap-northeast-1, 2026-10-10 (UTC), single AZ, c5n.9xlarge (36 vCPU), Amazon Linux 2023
• Mount: nfs4 vers=4.2, rsize=wsize=1048576, hard. The server is 127.0.0.1 (through efs-proxy, TLS and IAM authentication always on)
• Load: VDBENCH 5.04.07 (an I/O load generator distributed by Oracle), 512 threads, o_direct (bypasses the OS page cache)
• Data: a single uncompressible 600 GiB file. 60 seconds of warmup, then 300 seconds measured

Results: four workloads

Each row is one run; I did not repeat any condition. Source of this table: Results (S-1 + S-2) (Japanese)

Isolating the cause: efs-proxy CPU saturated at one core

I re-measured 1 MiB sequential reads for 60 seconds and sampled CPU three times.

The re-measured throughput was 1,109.4 MB/s, 0.4% away from the main run's 1,104.9 MB/s. The mount target is a local proxy at 127.0.0.1, and efs-proxy handles TLS termination and NFS translation in one process, so one mount tops out at one core of that process. Source of this table: Ceiling confirmation (Japanese)

Differences by instance type: three values under different conditions

Even on the same c5n.2xlarge the value changes with how I measured, so here are three side by side.

The three differ in conditions, so this is not a comparison that changes only the instance type. I have no measurement that varies only the instance type under the same conditions. I also did not isolate whether efs-proxy CPU or the client's network bandwidth sets the roughly 785 MB/s per c5n.2xlarge. Source of this table: Stream-count measurement (Japanese)

When one mount is not enough bandwidth, design for adding clients (the next chapter).

An aggregate that scales with client count: 8.01x at 8 clients

Going from 1 to 8 c5n.2xlarge clients raised the aggregate read from 784.4 MB/s to 6,282.6 MB/s (8.01x). Per client it stayed roughly constant at 784.4 to 802.2 MB/s. That fits the structure in which each client has its own efs-proxy.

Measurement conditions

• Environment: ap-northeast-1, single AZ, 2026-10-10 (UTC), up to 8 c5n.2xlarge, Amazon Linux 2023
• Mount: each client mounts S3 Files on its own (one efs-proxy per client)
• Load: VDBENCH, 1 MiB sequential read, o_direct, fixed at 64 threads per client (only the client count changes)
• Simultaneous start: shared epoch. All clients wait until the same UNIX time (90 seconds after launch) and start measuring together. 60 seconds of warmup, then 180 seconds measured
• Range read: two patterns, same (all clients read the same 600 GiB file) and disjoint (each client reads a non-overlapping range)

Results: all clients read the same file

Source of this table: Client-count measurement (same) (Japanese)

Results: each client reads a non-overlapping range

I measured disjoint at 8 clients only. The aggregate was 7,210.0 MB/s, and per client 901.3 MB/s (a spread of 770 to 1,332), higher than the 8-client same result of 6,282.6 MB/s. Each client reads different data, so sharing and fanning out one read cannot explain the number. I did not measure disjoint at 1, 2, or 4 clients, so the shape of disjoint scaling is unconfirmed.

Source of this table: Client-count measurement (disjoint) (Japanese)

Measurement limit: client-side totals only

The numbers are the sum of what the clients reported. I could not reconcile them with CloudWatch DataReadBytes: on the day, metrics arrived sparsely and did not land in the measurement window. The record also has no spread across repeated runs for each client count.

Figure 1: The internals of S3 Files. Each client has its own efs-proxy. A single mount tops out at one efs-proxy core, while the aggregate grows in proportion to client count (measured up to 8). The figure says the same thing as the tables above.

For capacity planning up to 8 clients, multiply roughly 785 MB/s per client (785.3 MB/s at 8 clients; c5n.2xlarge, all clients reading the same file) by the client count. I have not measured 9 or more. If you have a use case that shares it across 9 or more clients, tell me the conditions on the repository issue form, and I will measure from the conditions people use.

Comparing two ways to use the same data through two entry points: where the source of truth sits

Both S3 Files and FSx for ONTAP with S3 Access Points let you use the same data through a file mount and the S3 API. What differs is the source of truth: an S3 bucket for S3 Files, an ONTAP volume for FSx for ONTAP. That difference bears on the speed of propagation, how throughput grows as you add clients, and how the bill comes out. The measured values do not share an environment, though, so this post does not isolate whether the differences come from where the source of truth sits.

The two setups compared in this chapter are as follows.

• S3 Files: attach S3 Files to an S3 bucket and mount it over NFS from EC2. The S3 API reaches the bucket directly.
• FSx for ONTAP: use an ONTAP volume over NFS / SMB and attach S3 Access Points (the S3 API entry point attached to the volume) to the same volume. Data that does not fit on SSD can sit on the capacity pool, a tier with a lower storage price. Tiering All, one of the volume's tiering policies, places the data on the capacity pool.

The table below lines up the two setups by aspect. I wrote "not measured" where I did not measure. Measured values come from different environments, so each cell carries its environment.

The propagation rows are not a same-conditions comparison, because the NFS version, client, and configuration differ. The client-count rows differ too: S3 Files used VDBENCH with 64 threads per client, and FSx for ONTAP used a second-generation 2,048 MBps configuration with nconnect=16. I read in full on 2026-10-16 the two AWS documents this chapter relies on: the S3 User Guide pages Performance specifications and Understanding how synchronization works.

Source of this table: Design differences table. For the measured rows, the sources are the links in the "Source" column.

The four stages of file-to-bucket propagation, and how to treat the maximum reflection time

The "window of about 60 seconds" in the propagation rows is not one wait but four stages. The measured p50 of 63,769.3 ms is the whole span from (a) to (d).

The export side agrees across two sources. Against the client-side max of 66.33 seconds, the CloudWatch ExportAge peak was 66.06 seconds, a 0.27-second difference (all 30 runs fell in 63.8 to 66.3 seconds).

The maximum reflection time cannot be used as a guaranteed value. On the import side (S3 to file), the client-side max of 14.36 seconds and the CloudWatch ImportAge peak of 62.80 seconds do not agree. I could not find a maximum reflection time in the official documentation, so I do not treat it as an upper bound, and I write the representative value as the client-side p50. I do not write "reflected within 60 seconds", "reflected every minute even under continuous updates", or "never synchronized" either, because none has grounds. This explanatory model (wait, then aggregate, then export) is not an SLA.

Three notes follow.

• In a conflict (the same key changed from the file side and the S3 API side), the S3 bucket side became the source of truth (3/3 converged on the S3 content). The file-side version was moved to .s3files-lost+found-<fs-id> (matching the CloudWatch LostAndFoundFiles peak of 3.00).
• That directory is not created on conflict; it is present from the start (0700 root:root), and what happens on conflict is the eviction event. The official Synchronization page says files in this directory are not copied to the S3 bucket (I did not measure this).
• For large rename / move operations, a rename on the file side finishes immediately, while on the S3 side it becomes a per-object copy and delete. AWS says a rename of 100,000 files takes several minutes to reflect on the S3 side, and that the prefixes from before and after the rename are both visible midway. I have not measured this behavior separately.

Measurements that separate the propagation conditions (long continuous appends, after updates stop, while another file is being updated, concurrent updates) are deferred to a separate verification plan (a plan, not a new measurement).

What this comparison can settle is two things: whether the source of truth lives in an S3 bucket or an ONTAP volume, and whether you can accept the roughly 60 seconds from file to bucket. I estimate the monthly cost in the next chapter.

Monthly cost estimate: an order that splits by object size, and room for assumptions to move it

I apply the same list prices to the two setups from the previous chapter, with the stored volume fixed at 20 TiB. I estimated 10 cells: 5 patterns that differ in monthly read and write volume (P1, P4, P5, P6, P8) times 2 object sizes (64 KiB and 4 MiB). The amounts in this chapter are list price x quantity for ap-northeast-1, with prices read on 2026-08-09, and they are subtotals of the main line items. The subtotals add up the line items shown in the tables, and some items are not included. They are not measurements and not invoices. "Base assumptions" below means the values in the common-assumptions table in the next section, "Five patterns".

Under this model's base assumptions, in all 10 cells, the subtotal is lower on FSx for ONTAP at 64 KiB and lower via S3 Files at 4 MiB. At 64 KiB, FSx for ONTAP comes to $1,783.11 to $2,218.13 a month and S3 Files to $7,928.32 to $12,030.81. At 4 MiB, S3 Files comes to $512.14 to $543.52 a month and FSx for ONTAP to $1,781.28 to $1,788.08. Because the order was the same in all five patterns, what splits the order in this model is the object size, not the read and write volume. The candidate for the dividing line is the S3 Files import threshold (default 128 KiB). I estimated two sizes, 64 KiB and 4 MiB, so the tables have no rows near 128 KiB or at 1 MiB. The reading that the dividing line sits at the threshold comes from those two points and from how the price works (high-performance storage is charged only for sizes below the threshold), and I did not estimate sizes at the boundary. The import condition is sizeLessThan, so I treat exactly 128 KiB as not imported. This order is a result under the base assumptions, and it can flip when they change, so check the section "Assumptions under which the order can flip" in this chapter against your own values.

Five patterns: total monthly reads and writes

Writing "read N times a month" alone does not fix the total volume read. Whether the N reads cover all objects or only some of them changes the total several times over. I define each pattern with three numbers: the working-set ratio, the number of reads per working-set object, and the write volume. The table shows the monthly read and write volumes derived from them. The working set is the part that is read within the month. The working-set ratio is the working set's share of the stored volume, and the number of reads per working-set object is how many times each of its objects is read in a month. I set the write volume as a share of the stored volume (the churn rate).

Take P4 as an example. 10% of the 20 TiB stored (2 TiB) is the working set, and each of its objects is read 10 times a month. The volume read is 2 TiB x 10 = 20 TiB, which is 1.0x the stored volume. The other 90% (18 TiB) is not read at all within the month. With 4 MiB objects, that is 524,288 objects read 10 times each, or 5,242,880 reads. P4 has no writes, so the stored volume stays at 20 TiB.

Source of this table: The patterns

The placeholder values are numbers I put in to make the arithmetic work. They are not taken from a real workload. Replace them with your own monthly read and write volumes. The cost model also has P2, P3, and P7, but I left them out of the figures and tables because they have the same monthly read and write volumes, and so the same amounts, as P4 and P6 (P4 matches P2 and P3, and P6 matches P7).

The following assumptions are common to all 10 cells.

Source of this table: Common baseline

Cost figure 1: monthly cost of P1 and P4, which have no writes

Cost figure 1: breakdown of the monthly cost of P1 and P4, which have no writes (ap-northeast-1, list price x quantity, not an invoice). The horizontal axis is shared with cost figure 2. The working-set ratio and read count of P1 are placeholders with no source. "Figure 1" and "Figure 2" inside the images mean these two cost figures, not the internals figure in a preceding chapter. The figure shows the same numbers as the table below (dark theme version).

Source of this table: P1 / 64 KiB, P1 / 4 MiB, P4 / 64 KiB, P4 / 4 MiB

"Requests etc." is the sum of S3 requests, S3 Files read and write charges (metadata charges included), and FSx for ONTAP capacity pool read requests and PUT. It is the purple category in the figure. "Storage" for FSx for ONTAP is SSD and capacity pool storage added together.

The FSx for ONTAP rows assume reads are served from the file system, and they do not include read charges through S3 Access Points. The direct-read rows for the S3 API appear only for P1 and P4, which have no writes. In the patterns with writes, the write path differs by setup (S3 API PUT, NFS write plus sync, PUT through S3 Access Points), and putting them in one table would mix the write assumptions.

In the 4 MiB rows, the S3 Files subtotal is nearly the same as the direct S3 API read subtotal ($512.14 vs. $512.10 for P1, and $514.74 vs. $513.94 for P4). 4 MiB is at or above the threshold, so the data is not held in high-performance storage and is read straight from the bucket. In the 64 KiB rows, S3 Files adds $7,372.80 of high-performance storage.

Cost figure 2: monthly cost of P5, P6, and P8, which have writes

Cost figure 2: breakdown of the monthly cost of P5, P6, and P8, which have writes (ap-northeast-1, list price x quantity, not an invoice). The horizontal axis is shared with cost figure 1. The write volume of P5 and the working-set ratio, read count, and write volume of P6 and P8 are placeholders with no source. The figure shows the same numbers as the table below (dark theme version).

Source of this table: P5 / 64 KiB, P5 / 4 MiB, P6 / 64 KiB, P6 / 4 MiB, P8 / 64 KiB, P8 / 4 MiB

Adding writes adds, on the S3 Files side, PUT to the S3 bucket and, at 64 KiB, import writes to high-performance storage. On the FSx for ONTAP side it adds PUT through S3 Access Points. For P6 at 64 KiB, "Requests etc." is $4,146.01 via S3 Files and $436.88 on FSx for ONTAP. Across the three patterns of cost figure 2, the 4 MiB subtotals stay within $516.04 to $543.52 via S3 Files and $1,782.11 to $1,788.08 on FSx for ONTAP.

Items that push up the S3 Files subtotal at 64 KiB: high-performance storage and the number of writes

S3 Files imports files below the threshold (default 128 KiB) into high-performance storage and charges $0.36 / GB-month for the amount held. Files at or above the threshold are not held and are read straight from the bucket. The model assumes the whole stored volume (20,480 GiB) is active at the small size. That makes high-performance storage $7,372.80 in all five 64 KiB cells and $0.00 in all five 4 MiB cells.

On the FSx for ONTAP side, storage of $1,665.28 and fixed bandwidth of $115.97 are the same in all 10 cells. Only the "Requests etc." column moves with size and with read and write volume.

Why the S3 Files subtotal is higher at 64 KiB splits into two cases by pattern. In P1, P4, and P5, the S3 Files subtotal falls below FSx for ONTAP once the $7,372.80 of high-performance storage is removed. In P6 and P8 it stays higher even after the removal. Those two write 335,544,320 objects, and the S3 bucket PUT price ($0.0047 / 1,000) is higher than the PUT price through S3 Access Points ($0.00108 / 1,000), which also contributes. Those two PUT prices are still the values read on 2026-08-09 and were not part of the 2026-10-16 re-read.

The high-performance storage amount follows from assuming a 100% active share. That share is a placeholder, not a measured value. I made one observation of a related quantity (described in the section on assumptions under which the order can flip).

Metadata charges: the accounting assumption and one observation

The model counts S3 Files metadata as 4 KiB x operations, in separate import-side and read-side lines, and includes both in the subtotal. The column "Of which S3 Files metadata charges" in the tables is the sum of those two lines, and across all 32 cells it matches usage x unit price ($0.04 / $0.07). In two measurements, on 2026-10-16 and 2026-10-17, the Cost Explorer usage for APN1-Files-Read matched the sum of CloudWatch DataReadBytes and MetadataReadBytes to within 0.5% (daily, Estimated: true). Ingest was billed as APN1-Files-Write and did not appear in the CloudWatch write metrics. Because cost is set by usage (bytes), the per-operation breakdown of a read (open, getattr, and so on) does not change the estimate. The invoiced amount is offset to a net zero by credits, so I have not confirmed it as a billed figure. I did not measure other sizes or other instance types. The primary record of this reconciliation is at billing reconciliation.

I have one observation (2026-10-16, ap-northeast-1, m5.large, not reproduced). I read 1,000 4 MiB objects once each with dd, and CloudWatch MetadataReadBytes registered about 6.0 times per object, with an average of about 4.1 KiB each. Changing the I/O size to 1 MiB or 4 MiB made no difference. The 64 KiB control with the same procedure registered about 2.0 times. I do not know where the roughly 6 comes from, so I have not applied that count to the "4 KiB x operations" count in the model. DataReadBytes for 4 MiB had 0 data points, so I could not confirm whether the reads went through NFS or directly to the bucket. For the ingest of 4 MiB objects written through the S3 API, MetadataWriteBytes had 0 data points. I cannot tell whether no ingest happened or whether it is not metered.

The roughly 6.0 figure is the number of data points the CloudWatch metric recorded. I later queried Cost Explorer and reconciled it against the billed usage. The APN1-Files-Read usage on 2026-10-16 and 10-17 matched the sum of CloudWatch DataReadBytes and MetadataReadBytes to within 0.5% (daily, Estimated: true). I did not capture the breakdown of the roughly 6.0 operations, but because cost is set by usage (bytes), the breakdown does not change the subtotal.

The following remain unconfirmed:

• The invoiced amount itself. It is offset to a net zero by credits, so I have not confirmed it as a billed figure.
• Other instance types, nconnect, multiple processes, and other object sizes. I did not measure them.

The metadata column in the tables keeps the assumption of one operation per read. The difference in the operation count does not affect a cost that is set by usage.

Assumptions under which the order can flip: what to replace with your values

The order in this chapter is a result under fixed assumptions. If you replace the following assumptions with values from your environment, the amounts and the order move. The table of subtotals when one assumption moves at a time and the table of values at which the order flips are in the section "Baseline assumptions and sensitivity" of the generated document in the repository. A figure that takes P4 as an example and shows where each assumption sits and what kind of evidence backs it is in the section "Break-even map for pattern P4" (images: light and dark). Line length does not represent uncertainty.

Under the base assumptions, the FSx for ONTAP subtotal is lower at 64 KiB and the S3 Files subtotal is lower at 4 MiB. At 4 MiB, the direction does not change when one assumption moves at a time over the values tried, and it changes for every pattern in the scenario that moves several assumptions together toward a lower FSx for ONTAP subtotal (a combination that includes a value with no source). At 64 KiB, lowering the active share of S3 Files high-performance storage from the base makes the S3 Files subtotal lower, starting with the patterns that have fewer writes, though the base value of that share is a placeholder.

• Share of data on the capacity pool: fixed at 30% in every pattern.
• Storage efficiency of the SSD tier: fixed at 40%.
• Active share of S3 Files high-performance storage: fixed at 100%. A placeholder.
• S3 Files import threshold: the default 128 KiB.
• FSx for ONTAP deployment: Single-AZ first generation, 128 MBps.

There are also retention items that the base leaves out. Bucket versioning is required, so the S3 Standard storage charge for the non-current versions you keep is added to the S3 Files side. Keeping 1,024 GiB as an example raises the S3 Files subtotal by $25.60 a month in all 10 cells of the tables ($0.025 per GiB-month, model output). For snapshots on the FSx for ONTAP side, I have not confirmed which tier they occupy or how they are billed, so you can enter a capacity but it does not enter the subtotal.

I made one observation of a quantity close to the active share (2026-10-17, ap-northeast-1, 200,000 files of 64 KiB, the default import threshold). When I read 5% and 20% of the working set, the increase in StorageBytes, the billing dimension of high-performance storage, was 1.094 times the bytes read (5.47% and 21.89% of the stored volume). The working-set ratios are values the experiment set. StorageBytes is the sum of data and metadata, so it is not a direct measurement of the active share. In the same experiment ImportFailures registered about 200,000 per file system. I did not investigate the cause and did not separate its effect on StorageBytes. I have not confirmed the billed amount, so the ratio cannot be read as a dollar figure. It is one observation with no repeat in another environment, so I did not lower the 100% in the tables. The record is in the verification record (Japanese).

The capacity pool share and the active share of high-performance storage are fixed values decoupled from the working-set ratio. In P1, P4, and P5 the data touched within the month (the working set plus writes) does not reach the whole stored volume, so the two shares can be set again to match the volume actually touched. Fitting the active share of high-performance storage would lower the S3 Files subtotal at 64 KiB. For the FSx for ONTAP subtotal when the capacity pool share is fitted, the generated document only says that the two together would go down, and I have not checked each one separately. In P6 and P8 the data touched within the month adds up to the stored volume or more, so if the capacity pool share is assumed to cover the whole volume, the FSx for ONTAP subtotal would rise. The directions come from the generated document, and I have not calculated the change in amounts. One more point: for S3 Files at 64 KiB in P1 and P4, which have no writes, the model counts no first-read stream (the GET from the bucket and the import write into high-performance storage) and counts all reads as high-performance storage reads. Counting it would add some items and remove others, and the model does not settle the net direction.

The capacity pool read requests included in "Requests etc." for FSx for ONTAP assume one request per GET on the 30% of reads that hit the capacity pool. The actual count is unconfirmed.

An observation that helps comes from an adjacent repository (the FSx for ONTAP monitoring integrations). On a file system of the same type (Single-AZ first generation, 128 MBps, SSD 1,024 GiB), on 2026-10-11, I read volumes that had been created with ALL and written to directly, from a Lambda function outside the VPC. CloudWatch CapacityPoolReadOperations came to 0 per object at 64 KiB, about 5.4 to 5.6 at 1 MiB, and about 29.8 to 33.2 at 5 MiB (source: per-invocation reads, Japanese). The 1 MiB and 5 MiB values are for cases where operations registered, and 2 of the 8 runs built the same way registered none. The sizes measured were 64 KiB, 1 MiB, and 5 MiB; 4 MiB was not measured. What was counted is the CloudWatch operation count, not a billed count, and that record also leaves the relation between the two unconfirmed. The measurement date, the volume, and the client all differ from this post's measurements. For these reasons I did not apply these operation counts to the model's count (one request per GET). The capacity pool read request amount that follows from the count the model assumed is $0.03 to $1.16 across the five 4 MiB cells.

Reading data on the capacity pool: billing items exist, latency is unmeasured

Reading an object on the capacity pool of a Tiering All volume through S3 Access Points can incur two billing items: GET through S3 Access Points and capacity pool read requests (documented). This post's measurements did not produce the number of requests per GetObject or the latency of reading from the capacity pool. The capacity pool storage price is lower than SSD, $0.0238 vs. $0.15 per GB-month (Single-AZ, ap-northeast-1, read 2026-08-09).

These are the p50 observations of GET through S3 Access Points from a local machine.

The environment was 2026-10-10, ap-northeast-1, FSx for ONTAP Single-AZ first generation, 128 MBps, SSD 1,024 GiB, from a local machine outside AWS over the internet (round-trip time included), concurrency 1. It is one measurement: 166 to 407 requests on the SSD side and 10 requests after the change.

I cannot read this difference as a delay from reading the capacity pool, for three reasons.

• In the read window after the change, the per-volume CapacityPoolReadOperations was 0. The reads did not reach the capacity pool.
• The 5 MiB difference (+5.3 ms) is smaller than the spread from p50 to p90 on the SSD side (9.0 ms).
• I did not alternate SSD-side and post-change reads, and the request counts are asymmetric.

Source of this table: Reads by tier (Japanese)

Before you read much Tiering All data through S3 Access Points, look at the per-volume CapacityPoolReadOperations one-minute values in your own environment and then estimate.

Choosing between S3 bucket first and file system first

Laid out at the same granularity, the constraints of the two setups are as follows. For monthly cost, treat the rows as guidance under the base assumptions of the estimate above.

If your clients are Linux on AWS and you handle objects of 128 KiB or more while keeping the source of truth in an S3 bucket, start evaluating with S3 Files. If you have NFSv3 or SMB clients, read from outside AWS by mounting over NFS or SMB (I did not measure this), or mostly have objects below 128 KiB, start with FSx for ONTAP and S3 Access Points.

The object-size conditions are guidance drawn from the 64 KiB and 4 MiB results I estimated. I did not estimate sizes near 128 KiB. Decide the monthly order only after you replace the values in "Assumptions under which the order can flip" with those of your environment and check.

An invalid warm / cold comparison: lowering the threshold does not evict data already loaded

My first warm / cold comparison was invalid because both reads were warm. I have not confirmed the latency difference between warm and cold. Here warm means reading data held in high-performance storage, and cold means reading straight from the bucket.

I measured once with a different design (2026-10-19, ap-northeast-1, one c5n.9xlarge, Amazon Linux 2023, 64 KiB files placed one per directory, drop_caches before each read, default import threshold, concurrency 1). The p50 of a first read was 230.22 ms, and the p50 of re-reading the same file two minutes later was 18.71 ms (n=360 each). The paired difference on the same object was positive for all 360 pairs. But DataReadBytes was the same 65,972 B for the first and second reads, so I cannot distinguish the path of the first read. Of 40 objects in one directory, only the first was 438.9 ms, and the median of the other 39 was 13.12 ms (one observation; I did not investigate the reason). The conditions differ from the 4.05 ms (straight from the bucket) later in this chapter, and I did not investigate the cause of the difference. Details are in the verification record.

The goal was to check the claim that only frequently read data becomes low-latency. I assumed that lowering the import threshold (sizeLessThan, default: import files under 128 KiB) and then reading the same file would make the read cold, and I measured on that assumption.

Why it was invalid: the threshold only decides what the next first access imports

The threshold decides what gets imported the next time a directory is accessed for the first time. Data already in high-performance storage is not evicted when you lower it. Even in the window I meant to be cold, CloudWatch DataReadBytes (bytes read from high-performance storage) was nonzero, so both reads came from high-performance storage.

The numbers gave it away. The supposedly cold read was faster than the warm one, at 2.94 ms, which cannot be right. The record reads that difference as run-to-run variation and remount differences.

Three reads: two are valid

The environment was ap-northeast-1, 2026-10-10, c5n.9xlarge, 64 KiB, o_direct, concurrency 1, p50 of n=30.

Source of this table: Three reads (Japanese)

The two valid rows, 5.33 ms and 4.05 ms, differ by an amount within run-to-run variation. A separate t3.small record showed 10.8 ms from high-performance storage and 55.0 ms straight from the bucket (2026-09-01), but that did not reproduce on c5n.9xlarge. The environments and methods differ, so it is not a comparison that varies only the path in one environment.

To measure cold, read a new file that has not been imported after lowering the threshold, and confirm the path with DataReadBytes. The 2.94 ms in this post cannot serve as evidence for a warm / cold difference.

Status of the four stated values: I got numbers for two

Of the four values AWS states, I got numbers for two: propagation time when using both files and the S3 API, and aggregate growth with client count (up to 8). The comparison for "only frequently read data becomes low-latency" was invalid, so I cannot confirm it. I did not measure "existing tools work as they are".

Sources of this table: Propagation measurement, Three reads, Client-count measurement (same) (Japanese)

The values come with costs. Propagation from file to bucket takes about 60 seconds, the supported protocols are only NFSv4.1 and NFSv4.2, bucket versioning is required, and clients are limited to Linux on AWS. The two confirmed points are usable as grounds for adoption. Treat the low-latency value as an assumption only after you measure a cold read in your own environment.

What I have not measured

This post's measurements cover a single S3 Files mount (c5n.9xlarge), S3 Files client count (c5n.2xlarge, 1 to 8 clients), S3 Files 64 KiB cold and warm latency (one client), and latency by tier (SSD vs. capacity pool) through FSx for ONTAP S3 Access Points for Single-AZ and the preferred AZ of Multi-AZ first generation, inside a VPC. I did not measure the following four things.

If any row has conditions that differ from yours, measure that row first.

Reproducing it yourself: three runbooks from creation to teardown

You can follow this post's measurements in your own AWS account with the same steps. I put runbooks from creating the environment to tearing it down on GitHub. The links are pinned to the commit of the measurement.

• Create the environment and mount: S3 Files verification environment setup (Japanese). It builds with CloudFormation and runbook.sh. The most frequent cause of a silently hanging mount (missing CloudWatch permissions on the host role) is written at the start of these steps.
• Measure throughput and client count: from the section "Measuring throughput and client count" on the same page, go to throughput-runbook.sh (single mount) and s4-ladder.sh (1 to 8 clients). VDBENCH requires accepting Oracle's license, so obtain it separately.
• Results and conditions: S3 Files file system performance (Japanese). This is where this post's numbers come from, each with its environment. It also states that each is a single sample.

Leaving the verification environment up keeps billing. When you finish measuring, tear it down with the runbook's destroy and stop, then confirm by count that nothing remains.

Closing

Two things hold from the measurements, and one from the estimate.

• The single-mount ceiling was one efs-proxy CPU core. It was 1,104.9 MB/s for 1 MiB sequential reads on a c5n.9xlarge (one run, 2026-10-10, ap-northeast-1).
• When bandwidth falls short, adding clients raised it. From 1 to 8 c5n.2xlarge clients, the aggregate went from 784.4 MB/s to 6,282.6 MB/s (8.01x). I have not measured 9 or more.
• In the estimate, under this model's base assumptions, the monthly cost order split by object size, not by read and write volume. The candidate for the dividing line is around the S3 Files import threshold (128 KiB), and I estimated only two sizes, 64 KiB and 4 MiB. With the stored volume fixed at 20 TiB and five patterns (P1, P4, P5, P6, P8), 64 KiB comes to $1,783.11 to $2,218.13 a month on FSx for ONTAP and $7,928.32 to $12,030.81 via S3 Files, and 4 MiB comes to $512.14 to $543.52 via S3 Files and $1,781.28 to $1,788.08 on FSx for ONTAP (ap-northeast-1, subtotals of the main line items with prices read on 2026-08-09; list price x quantity, not an invoice). This order is a result under the model's base assumptions and can flip when they change, so check it against your own values.

Which one to consider first narrows down by where the source of truth lives and where clients read from. Because the monthly order by object size can change with assumptions, treat the object-size conditions in the next table as guidance under the base assumptions of the estimate.

I keep every measured value, each with its environment, on GitHub. I hope this helps anyone with a similar need (collect in S3, read as files). <!-- allow:ai-style the "I hope this helps" closing mirrors the JA 一助になれば幸いです and is the author's fixed first-person sign-off (global-author-voice); HTML comments are stripped on publish so the visible text is unchanged -->

Until next time.

Yoshiki Fujiwara