Async PyMongo in FastAPI

This tutorial was written by Gourav Bhabesh.

1. Introduction

Why modern APIs need asynchronous I/O

Modern APIs need to be robust and be able to serve calls on the go, and minimize the instances of timeouts and server overloads while providing a good user experience during long-running jobs. Asynchronous I/O can solve one of the key issues making modern APIs: staying fast and reliable. It lets the user know the progress of their task while the API handles the requests in the background and is available for other requests. One of the key challenges in the API calls to the database is that some queries can be returned in a few milliseconds but others can take seconds (even with well indexed queries). Without asynchronous I/O in the API, this can lead to long request queues.

You can imagine this as a bakery storefront serving different goods at the same time to multiple customers. We want the orders to be filled asap without the queue becoming too big. In our example, we have multiple servers (scalable infrastructure) who take the orders which can take varying time to process and each customer with longer order times is given a status bar to know where their order’s current status. So a customer with an order that can be instantly filled (picking up a cookie) does not have to wait behind such a customer with a long processing order (baking a cheese cake).

FastAPI’s design philosophy around async

The building block of the FastAPI is ASGI (Asynchronous Server Gateway Interface) standard, unlike the older WSGI (Web Server Gateway Interface) which was synchronous in nature by default. In short, it uses Python’s async/await syntax, yielding control to the event loop, allowing servers to process other requests in the meantime, rather than blocking an entire thread. Using an event loop lets FastAPI handle thousands of concurrent requests with fewer resources. This leads to higher throughput per thread and lower latency. This automatically encourages the developer to use the async ecosystem to fully leverage the performance benefits of async programming.

Database bottlenecks & why async DB drivers matter

One of the key challenges that any database faces is the uncertainty in response times, in which the queries will be answered by the database server. Some queries can take milli-seconds and some can take seconds. If the architecture is not well built to handle this uncertainty, it can cause severe database bottlenecks. To handle this challenge, async database drivers ensure that longer requests do not become a bottleneck for other incoming requests. Initially, the async MongoDB driver was built in the library Motor. But in late 2023, Pymongo team officially decided to build the native async solution, with the goal to replace Motor as the long-term solution.

Introducing the new Async API in PyMongo

The Pymongo team has been working on the goal to replace the Motor library to give native async support in the pymongo library itself. This gives confidence on the client side to use the official library with async support. The async support was first provided in the version 4.9 in Pymongo and it has been further stabilised to the current version of 4.15.0. The new API capabilities bring the asyncio support directly to the official MongoDB driver, allowing developers to use awaitable database operations without external libraries like Motor. Combined with the FastAPI framework, which already supports await/async capability, Pymongo can now be natively used to build high-throughput, non-blocking services.

2. What is PyMongo?

Overview of the official MongoDB Python driver

MongoDB has provided Pymongo as the official Python driver to allow the applications to communicate with MongoDB clusters, providing production-tested API to perform CRUD operations, manage indexes, run aggregation pipelines and handle advanced features such as transactions and change streams. Pymongo was traditionally built on a synchronous I/O model, meaning every database call made will block the executing thread until the MongoDB server does not return a response leading to database bottlenecks.

Traditional synchronous PyMongo behavior

The key behaviours of the traditional Pymongo library, which we should be aware of, to appreciate the addition of the native support of asynchronous, are that synchronous operations block Python threads until the operation completes or a network error or timeout occurs.. This leads to another behaviour - sequential execution, blocking all requests until and unless the previous request is not complete. This made the MongoClient “Thread-Safe” allowing for multi-threading applications to be written, but each thread still performs blocking database operations.

In our example, of the baking shop, it would be like each server is concurrently serving the customers but each server is waiting for the order to complete rather than handing a token to wait, while serving the next customer request which might be served in much less time.

Blocking vs non-blocking I/O explained simply

Blocking I/O means the application must wait for the database operation to finish or network timeout before it will allow anything else to continue. On the other hand, non-blocking I/O allows requests to continue flowing to the database while previous requests are still being processed. In addition, the async drivers provide a way to receive the status of the request, giving power to the developer to handle long operations and create a better user-experience.

Limitations of synchronous DB operations in async frameworks

Using synchronous DB operations in the async framework is one of the key mistakes made by many clients of MongoDB, part of the reason was the non-availability of official Python drivers that actually supported asynchronous behavior. If the synchronous Pymongo driver is used inside an async framework such as FastAPI, these blocking calls freeze the event loop, preventing the server from processing other incoming requests and significantly reducing concurrency. As a result, applications that rely on synchronous database drivers inside async frameworks often experience increased latency, request queuing, and poor scalability under load, even if the rest of the code is written using async and await.

Imagine our baking shop upgraded its entire framework to be non-synchronous. The cooks can cook multiple dishes at the same time, but the servers are still the same, processing sequential orders and not taking new orders until the previous order has been delivered. The cook is having an easy day!!

3. What is Async PyMongo?

Overview of PyMongo’s new async API (pymongo.async_)

The official MongoDB team started working on the native-async support in the Pymongo driver in version4.9, where it was first released as beta version in late 2023. As of writing this article, the pymongo version 4.15 has been released and has been stable since version 4.13. With the pymongo.With async support, we can create an AsyncMongoClient object similar to the MongoClient object we created in the previous synchronous call. The Client provides fully awaitable database operations such as insert_one, find_one, update_one, and aggregation pipelines, allowing MongoDB I/O to integrate natively with Python’s event loop.

Key advantages over synchronous API

Compared to the traditional synchronous API, async PyMongo significantly improves scalability and responsiveness under concurrent workloads. Because database calls no longer block the event loop, asynchronous framework applications can continue handling new requests while waiting for MongoDB responses, resulting in lower latency, higher throughput, and better CPU utilization—especially in high-traffic microservices

Relationship to Motor (Motor’s role before async PyMongo existed)

Before async Pymongo, motor was the de facto library to be used when developers needed asynchronous behavior when communicating with the MongoDB clusters. Motor did this by exposing a coroutine-based API, allowing applications to switch context and perform other tasks while waiting for I/O operations. However, Motor was not a separate driver from scratch; it was designed as a wrapper around the core synchronous PyMongo library. It used Tornado’s or asyncio’s event loop, wrapping Pymongo’s operations asynchronously, effectively allowing the main event loop to remain unblocked. Motor was specifically created to support asynchronous web frameworks, initially Tornado and later adding support for asyncio. Performance comparison between motor and async pymongo can be found on the official documentation here.

How to migrate from motor to async-pymongo and sync-pymongo to async-pymongo

Migrating from the motor-based asynchronous library to the async-pymongo library is simple and is well documented by MongoDB here. In simple terms, one needs to change the driver call to the database using the AsyncMongoClient and add the keyword await in front of the database operations.

When migrating from sync Pymongo to async Pymongo, keep the following important points from the official MongoDB documentation in mind:
· To convert an AsyncCursor to a list, you must use the asynchronous cursor.to_list() method.
· The AsyncCollection.find() method in the PyMongo Async API is synchronous, but returns an AsyncCursor.
To iterate through the cursor, you must use an async for loop.
· The AsyncMongoClient object does not support the connect keyword argument.
· You cannot share AsyncMongoClient objects across threads or event loops.
· To access a property or method of a result returned by an asynchronous call, you must properly wrap the call in parentheses, as shown in the following example:

id = (await posts.insert_one(doc)).inserted_id

4. Key Features of Async PyMongo

True async interface with awaitable operations

The biggest advantage of the async Pymongo library for developers is its awaitable operations. Functions such as insert_one(), find_one(), update_one() and aggregate() return objects that can wait rather than blocking the results. This fundamentally decouples the event loop from the operation, meaning the database operation running on the database side yields the control back to the event loop, allowing FastAPI to process the other incoming requests without blocking the thread.

In our baking shop example, the server has been decoupled from the chef, while the previous order is getting cooked in the kitchen, the server takes in a new order without ceasing the operation and blocking the thread and allowing for incoming requests to be fulfilled !! This model of working is fundamentally different from the traditional synchronous PyMongo, where every database call ties up a worker thread until completion, blocking the thread from serving the next call when under load.

Connection pooling behavior

The connection pool behaviour is quite an interesting one here. On the surface it appears both the synchronous and the asynchronous Pymongo connection-poling implementation look the same (parameters available - maxPoolSize, minPoolSize, waitQueueTimeoutMS etc.) and indeed rely on the same MongoDB C driver. However, async applications are actually able to spawn thousands of concurrent operations and grab many connections from the pool at once, whereas, in the synchronous application each request occupies a python worker thread, holding a connection from the pool waiting for MongoDB. This naturally limits concurrency as the application is limited by the number of workers, effectively meaning the full pool size can be never used optimally.

Imagining in our world of baking shop, we have an army of chefs to process the orders, the async model allows the server to grab hold of any available chef to accept a new order and move on to the next one, whereas in the synchronous setup the server will wait for the chef to finish the previous order before moving to the new order.

Async cursors and non-blocking iteration

One of the primary challenges that any database faces is returning results to query with a large result set. The traditional PyMongo driver already returns a cursor object rather than loading all the documents into the memory at once. In async PyMongo, these cursors are fully asynchronous in nature as well, resulting in iteration without blocking the event loop. Developers now have the choice of using the await cursor.to_list() or iterating over the cursor object using the async for. This will allow data to be streamed in batches while the application is handling incoming requests. This feature is of significant importance when we have endpoints that return large datasets, background processing jobs and real-time analytics workloads.

5. What is FastAPI?

FastAPI’s async-first architecture

FastAPI is a high-performance web framework that is production-ready and designed for building APIs in python, with roots embedded in asynchronous programming. It embraces Python’s asyncio model as its core principle rather than treating it as optional. This design choice gives FastAPI leverage for I/O bound workloads such as db access, external API calls and real-time data streaming.

Other traditional web frameworks default to synchronous request handling, whereas FastAPI encourages using asynchronous endpoints (async def) wherever possible. When an endpoint is declared using the async keyword, FastAPI uses the event loop execution rather than a worker thread. Now as long as those requests involve the non-blocking I/O operations, a single worker process can handle many requests concurrently. This is where FastAPI’s async-first architecture shines. When the entire request path - including the database access - is non blocking, each request becomes super efficient. Any stack in the request that blocks the event loop will make the FastAPI’s concurrency model diminish significantly.

Event loop and concurrency model

At the heart of FastAPI’s concurrency model is Python’s asyncio event loop. The event loop manages the execution of many coroutines, switching between them whenever one is waiting for I/O, such as a network response from MongoDB. This cooperative multitasking enables a single process to efficiently handle thousands of simultaneous requests without spawning thousands of threads. However, this model only works as intended when all long-running operations are awaitable. If a synchronous operation blocks the event loop — for example, a traditional synchronous PyMongo database call — the entire server becomes unresponsive to other requests until that operation completes.

Why FastAPI pairs naturally with non-blocking database drivers

As explained above the core principle of FastAPI is to yield the control back to the event loop and any blocking I/O operation in the chain of the operations sent by the request will be like throwing a wrench on the gears. With the pairing of the non-blocking db driver such as the async pymongo, when a FastAPI endpoint awaits a MongoDB operation, control is yielded back to the event loop, allowing for the next set of requests to be processed in parallel. With the alignment between the db driver and the base principle of FastAPI to be async, we eliminate the friction and make the full request truly asynchronous in nature and hence enable FastAPI to scale efficiently under high concurrency.

Surprising performance differences when mixing sync DB code into async routes

A common pitfall for developers writing code in FastAPI is mixing the synchronous PyMongo calls inside the async def endpoints. On the surface it may appear to work, but it breaks the core principle of the FastAPI to be truly asynchronous. One single blocking db call and the entire event loop gets stalled as other requests will have to queue behind it - even when the server has many workers available.

In practice, this means that an API written with FastAPI but using synchronous PyMongo can perform worse under load than a fully synchronous application with multiple worker threads. By contrast, pairing FastAPI with Async PyMongo preserves the framework’s intended concurrency benefits, leading to better throughput, lower latency for typical requests, and more predictable scaling behavior. In large applications, having the capability to predict the scalability of infrastructure and have a control on the concurrency is of paramount importance.

6. Why Use Async PyMongo with FastAPI?

Avoid event-loop blocking

Using FastAPI with Asynchronous Pymongo correctly is essential if we want to avoid event-loop blocking. The requests from the async-first framework like FastAPI, share the same event-loop within each worker process. Meaning any operation in this request if becomes synchronized will lead to blocking of the entire event-loop, which high-concurrency systems want to avoid at all cost.

By using Async Pymongo, each db operation becomes awaitable rather than blocking in nature, ensuring a slow db operation does not freeze the entire application, allowing other requests to get processed.

Handle thousands of concurrent requests

FastAPI is capable of handling thousands of concurrent requests per worker process, but designing the entire chain of operations in an asynchronous manner is a must to achieve this. One of the primary mistakes that are made in the coding side is missing the async and await keywords in the db operation call, this breaks the core principle of FastAPI leading to slowness. With correct implementation we can make the infrastructure more cost-effective and efficient as compared to a purely synchronous system which would not be scalable in the long run as more threads are required to handle the huge volume of requests.

Reduce latency under load

With modern tech stacks, both horizontal and vertical scaling is a must and systems built with such capabilities can withstand the test of time. And today's system should be capable enough to handle varying loads. With FastAPI - async nature is built in-core principle of the library itself and when combined with Async Pymongo it allows for the potential blocking db operations to be handled asynchronously. This leads to lower median latency for requests, especially I/O intensive ones. In section 8, we will see how the sync/async jobs specifically prove this point, when we compare the median latency of thousands of concurrent requests.

Improve throughput for high-traffic microservices

In the world of microservices where responsiveness and scalability are critical Async Pymongo dramatically improves the performance by removing the blocker of such requests to the db layer. Combined with async frameworks such as FastAPI, Async Pymongo is superbly placed to serve real-time applications, analytics dashboards or user-facing services.

Better CPU utilization and scaling on limited infrastructure

One of the vital nerves of any IT infrastructure is cost-efficiency and with Devops team’s mantra - “doing more with less”, fits perfectly with the Async Pymongo. Making the most of the available resources, async architecture uses the same thread without blocking incoming requests by handing over the control back to the single event loop. Instead of maintaining hundreds or thousands of idle threads waiting on the I/O, FastAPI with Async Pymongo exploits this single event loop allowing it to switch between tasks as needed. In cloud based infrastructure, where compute resources are directly tied to cost, it can lead to meaningful savings without sacrificing performance.

7. Async CRUD Operations with Examples

This section demonstrates the common MongoDB operations asynchronously using the pymongo.async_ inside FastAPI.

All examples assume:

from pymongo.async_ import AsyncMongoClient 
from bson import ObjectId 
client = AsyncMongoClient (MONGO_URI) db = client["appdb"] 
users = db["users"]

Insert one / insert many (async versions)

Insert one document example:

@app.post("/users") 
async def create_user(user: dict):
  result = await users.insert_one (user) 
  return {"inserted_id": str(result.inserted_id)}

Insert many documents example:

@app.post("/users/bulk") 
async def create_many(users_in: list[dict]): 
  result = await users.insert_many (users_in) 
  return {"inserted_ids": [str(i) for i in result.inserted_ids]}

Find one / find many with async cursors

Find one document example:

@app.get("/users/{email}")
async def get_user(email: str): 
 return await users.find_one ({"email": email})

Find many documents example:

@app.get("/users") 
async def list_users():
  cursor = users.find ({}) 
  return await cursor.to_list(length=100)

Async pagination using cursors

@app.get("/users/page") 
async def paginated_users(last_id: str | None = None, limit: int = 20): 
  query = {"_id": {"$gt": ObjectId(last_id)}} if last_id else {} 
  cursor = users.find(query).sort("_id", 1).limit (limit) 
  results = await cursor.to_list(length=limit) 

  return { 
    "results": results,
    "next_cursor": str(results[-1]["_id"]) if results else None, 
  }

(Avoid skip() for large collections; use _id cursor pagination instead.)

Update and replace operations

Update one document example:

@app.patch("/users/{email}")
async def update_user(email: str, changes: dict):
  result = await users.update_one(
    {"email": email},
    {"$set": changes},
  ) 
  return {"matched": result.matched_count, "modified":
result.modified_count}

Replace one document example:

@app.put("/users/{email}")
async def replace_user(email: str, new_doc: dict):
  result = await users.replace_one({"email": email}, new_doc) 
  return {"matched": result.matched_count}

Delete one/many operations

Delete one document example:

@app.delete("/users/{email}") 
async def delete_user(email: str):
  result = await users.delete_one ({"email": email}) 
  return {"deleted": result.deleted_count}

Delete many documents example:

@app.delete("/users/inactive")
async def delete_inactive():
  result = await users.delete_many ({"active": False}) 
  return {"deleted": result.deleted_count}

Async aggregation pipelines

@app.get("/users/stats")
async def user_stats():
  pipeline = [
    {"$group": {"_id": "$country", ""count"": {"$sum": 1}}},
    {"$sort": {"count": -1}},
  ] 
cursor = await users.aggregate (pipeline)
return await cursor.to_list(length=100)

Error handling (duplicate keys, timeouts, network issues)

Duplicate keys:

from pymongo.errors import DuplicateKeyError
from fastapi import HTTPException

try: 
  await users.insert_one({"email": "a@b.com"})
except DuplicateKeyError:
  raise HTTPException(status_code=400, detail="User already exists")

Timeout/network issues:

from pymongo.errors import ExecutionTimeout, ConnectionFailure

try:
  await users.find_one({"email": "a@b.com"})
except ExecutionTimeout:
  raise HTTPException(status_code=504, detail="DB timeout")
except ConnectionFailure:
  raise HTTPException(status_code=503, detail="DB unreachable")

8. Performance Comparison: Sync PyMongo vs Async PyMongo

In this section, we break down the high-stakes battle between the traditional Synchronous (Sync) driver and the modern Asynchronous (Async) driver using the large-scale sample_airbnb dataset. This isn't just a syntax preference; it's the difference between a server that remains responsive under fire and one that collapses under its own weight.

Example endpoints and profiling results

To push these drivers to the limit, we implemented a Complex Aggregation task. Instead of a simple "Ping," we forced each request to filter high-rated Airbnb listings, sort them by price, and project complex nested fields.

Sync Logic: Utilizes list(users.aggregate(pipeline)), which occupies a physical thread for the duration of the database round-trip and data serialization.

Async Logic: Utilizes await (await users.aggregate(pipeline)).to_list().
The server yields control during the database "wait time," allowing the single-threaded event loop to process other incoming users.

Locust Benchmark setup

We subjected both implementations to a "Trial by Fire" using Locust:

User Load: Scaled up to 500 concurrent users.

Hardware Constraint: We intentionally limited the Sync thread pool to 1 thread to simulate extreme resource exhaustion and observe the "Queueing Effect."

Target: listingsAndReviews collection in MongoDB Atlas.

Understanding event-loop blocking

The results highlight a fundamental architectural divide. In the Sync model, when the thread pool is exhausted (in our case, limited to 1), every new user must wait for the person in front of them to finish their entire database operation. This creates a "Single Lane Bridge" bottleneck. In the Async model, even with thousands of users, the event loop acts as a high-speed scheduler. It sends the request to MongoDB and immediately moves to the next user, effectively "stacking" thousands of concurrent database operations without needing a thread for each one.

Graphs and metrics discussion (Locust latency plots)

Table: Sync stats showing the performance on one thread
Graph: Sync response times

Table: Async stats showing the performance

Graph: Async response times

Looking at our Locust metrics, the disparity is stark:
| Metric | Sync (1 Thread) | Async (Event Loop) |
|-------------------|-----------------|--------------------|
| Max RPS | ~48.8 | ~113.4 |
| Median Latency | ~8,700 ms | ~2,800 ms |
| 95th-Percentile | ~9,400 ms | ~4,100 ms |
| Throughput Driver | Thread-bound | DB / Network-bound |

Sync Graphs: Notice the staircase effect in response times.
As users increase, the latency climbs linearly because users are waiting in a queue. The RPS is "hard-capped" because the single thread can only process so many documents per second.

Async Graphs: The response times remain significantly lower and more stable.
Even as we ramped to 500 users, the 95th percentile hovered around 4 seconds compared to nearly 10 seconds for Sync. This proves that Async manages high-concurrency significantly better, even when the individual database query is complex.

Full working snippet for readers to run

Ready to see these results on your own machine?
We have provided a complete repository containing the FastAPI implementation, the AnyIO thread limiter configuration, and the Locust scripts used for these benchmarks.

Demo work - https://github.com/gouravmongodb/fastapi_pymongo_demo

9. Common Pitfalls & How to Avoid Them

Even with Async PyMongo, it is easy to introduce subtle performance or correctness issues if you are not careful. Below are the most common pitfalls developers encounter and how to avoid them.

Forgetting await on Database Calls

Because Async PyMongo methods are awaitable, forgetting await will not raise an immediate syntax error but will return a coroutine object instead of executing the operation. This can lead to silent failures or unexpected behavior in your API responses. Always double-check that every database operation inside an async def route is properly awaited.

Creating a New Client for Every Request

Instantiating AsyncMongoClient inside each request handler is a major anti-pattern. It creates unnecessary TCP connections, increases latency, and quickly exhausts resources under load. Instead, create a single shared client at application startup (using FastAPI lifespan or startup events) and reuse it across all requests.

Returning Non-JSON-Serializable Types

MongoDB types like ObjectId, datetime, or Decimal128 are not directly JSON serializable. Returning raw MongoDB documents from FastAPI will cause serialization errors. The recommended approach is to either convert these types to strings or map your documents to Pydantic models that handle serialization correctly.

Exhausting Connection Pools Under Heavy Load

Async applications can issue far more concurrent database operations than synchronous ones, which makes connection pool tuning critical. If maxPoolSize is too low, requests will start queuing inside the driver, increasing latency. Monitor Atlas metrics and adjust maxPoolSize, minPoolSize, and maxConnecting based on your workload.

Mixing Synchronous and Asynchronous Code

Using synchronous PyMongo calls inside async def routes (or vice versa) undermines FastAPI’s concurrency model and can block the event loop. Choose one model—preferably Async PyMongo for FastAPI—and stick to it consistently throughout your application.

10. Conclusion

Asynchronous database access is a cornerstone of modern, scalable API design. In an async-first framework like FastAPI, using a non-blocking database driver is not a luxury—it is a necessity for handling high concurrency, reducing latency, and making efficient use of resources.

With the stabilization of pymongo.async_ in PyMongo 4.13, Async PyMongo has become the recommended and officially supported way to use MongoDB with FastAPI. It provides a native, awaitable interface that aligns with FastAPI’s event loop, simplifies dependencies, and brings the future of the MongoDB Python ecosystem under a single driver.

Synchronous PyMongo still has a place in the ecosystem — particularly for simple scripts, batch jobs, or applications that are not built around async frameworks. However, for production web services, real-time applications, and microservices that must scale, Async PyMongo is the better choice.

Ultimately, performance depends not just on “sync vs async,” but on how you design your workload, tune your connection pool, index your collections, and scale your MongoDB cluster. When used thoughtfully, Async PyMongo with FastAPI offers a powerful, maintainable, and future-proof foundation for building high-performance Python APIs.

Research work

· https://pymongo.readthedocs.io/en/stable/changelog.html#changes-in-version-4-9-2024-09-18
· From Pymongo driver v4.9.0 introduced the beta version of async function.
· From Pymongo driver v4.13.0 - asynchronous API is now stable and no longer in beta.
· https://www.mongodb.com/docs/languages/python/pymongo-driver/current/reference/migration/#migrate-to-pymongo-async
· https://www.mongodb.com/docs/languages/python/pymongo-driver/current/reference/migration/#performance-benchmarks