Stop Killing Your Database With N+1
Cover photo by Taylor Vick on Unsplash
N+1 queries are the silent killer of application performance.
If you have a loop that executes a database query for every item in a list, you are doing it wrong. Let us say you are fetching a list of ten authors and then fetching their books. This pattern results in one query to get the authors, plus ten separate queries to get the books for each one. That is eleven queries total. If you have one thousand authors, your application hits the database one thousand and one times. That is the cost of N+1.
Why it happens
It happens because ORMs make it too easy to be lazy. You see code like this in many tutorials:
const authors = await db.authors.findMany();for (const author of authors) { const books = await db.books.findMany({ where: { authorId: author.id } }); console.log(books);}It looks clean. It reads like plain English. But it executes on the wire as a blizzard of tiny, unnecessary round trips. Your database server is likely sitting right next to your app server, so the latency feels negligible during development. Once you hit production, that latency compounds. Network round trips add up fast.
The fix is simple
Instead of fetching books inside the loop, use a join or an ‘in’ clause. Most modern tools support eager loading or including related data in a single pass.
// Do this insteadconst authorsWithBooks = await db.authors.findMany({ include: { books: true }});This turns your N+1 problem into two queries regardless of the count. One for the authors, one for all the relevant books. I realize there are cases where eager loading consumes too much memory if the related data set is massive (and yes, loading a million rows into memory at once is a different kind of disaster), but for 99 percent of standard CRUD apps, eager loading is the correct default.
Is it always wrong?
“But what if I only need the books for two out of the ten authors based on complex logic?”
You are right. There are times when eager loading everything is inefficient. In those specific, rare cases, fetching per item might actually save memory. However, I have seen developers reach for that justification to hide sloppy code far more often than I have seen them use it to solve a legitimate performance bottleneck. Don’t optimize for the edge case before you have a measurement proving you need to.
How to catch it
Turn on query logging in your development environment. If you see your terminal window scrolling endlessly with identical looking ‘SELECT’ statements while a page loads, you have found your problem. Don’t look at the code. Look at the network waterfall. If you see a staircase of queries, you know what to do.
Are you checking your query logs regularly, or are you just shipping and praying?