Why Retries Are Killing Your API

The Retry Trap

We all want our applications to be resilient. When an API call fails, the first instinct is to try again. It sounds simple. If the network blips, a second attempt usually succeeds. But this is where good intentions lead to system-wide outages.

The Thundering Herd Problem

Say a microservice is struggling under load. It starts returning 500 errors. If you have ten thousand clients configured to retry immediately, you are essentially launching a distributed denial of service attack on your own infrastructure. This is the thundering herd. By adding more requests to a service that is already failing, you ensure it stays dead.

Naive Retries

This is what you want to avoid:

1
async function fetchData(url, retries = 3) {
2
  for (let i = 0; i < retries; i++) {
3
    try {
4
      return await fetch(url);
5
    } catch (e) {
6
      console.log('Retrying...');
7
    }
8
  }
9
}

This code tries to reconnect as fast as the CPU allows. It does not respect the state of the server. If the server is overloaded, this loop just pours gasoline on the fire.

Implement Exponential Backoff

Instead of slamming the server, wait longer between each attempt. If the first retry waits 100ms, the next should wait 200ms, then 400ms, and so on. This gives the downstream system breathing room to recover.

1
const delay = (ms) => new Promise((res) => setTimeout(res, ms));
2

3
async function fetchWithBackoff(url, retries = 3, backoff = 100) {
4
  for (let i = 0; i < retries; i++) {
5
    try {
6
      const response = await fetch(url);
7
      if (response.ok) return response;
8
    } catch (e) {
9
      // Log the error
10
    }
11
    await delay(backoff * Math.pow(2, i));
12
  }
13
  throw new Error('Max retries reached');
14
}

Adding Jitter

Even with backoff, if your clients are synchronized, they might all retry at the exact same millisecond. This creates waves of traffic. To fix this, add jitter. By adding a random amount of time to your delay, you spread out the traffic load, which prevents the waves of requests from hitting the server at once.

1
const jitter = Math.random() * 100;
2
await delay((backoff * Math.pow(2, i)) + jitter);

When To Stop Retrying

Not every error deserves a retry. If you get a 401 Unauthorized or a 404 Not Found, retrying will never help. Only retry on transient failures like 503 Service Unavailable or network timeouts. Retrying on business logic errors is just a waste of compute and bandwidth.

Final Thoughts

Resilience is not about ignoring errors, it is about handling them gracefully. If you must use retries, always use exponential backoff and add jitter. Otherwise, you are not fixing your downtime, you are guaranteeing it.