Automating Database Backups With AWS Lambda

You don’t need a complex pipeline to reliably back up your production database. Many people swear that you need a dedicated server running cron jobs for backups. That’s only true if you have massive, multi-terabyte datasets that take hours to stream. For most applications, a simple function is more than enough.

The myth of the dedicated backup server

People assume that because databases are heavy, they require heavy infrastructure. It makes sense on the surface. If you have a script running on a Linux box, you have control. But you also have to manage that box (which is usually just overhead I’d rather avoid). If that server goes down, your backups stop. You’ve just created a new single point of failure.

Using Lambda for the heavy lifting

AWS Lambda is great for this because it’s ephemeral. You spin it up, dump your database to S3, and it vanishes. Here is the basic structure of a Python script to handle a Postgres dump.

1
import subprocess
2
import boto3
3

4
def lambda_handler(event, context):
5
    # This is a bit simplified, but you get the idea
6
    cmd = ["pg_dump", "-h", "db_host", "-U", "user", "dbname"]
7
    with open("/tmp/backup.sql", "w") as f:
8
        subprocess.run(cmd, stdout=f)
9

10
    s3 = boto3.client("s3")
11
    s3.upload_file("/tmp/backup.sql", "my-backup-bucket", "backup.sql")

(I’m assuming you’ve got your binary dependencies like pg_dump bundled into a Lambda Layer, because otherwise, this won’t work.)

Is this really safe?

“But what about memory limits and execution timeouts?” That is a fair point. If your database is huge, your Lambda will timeout or run out of memory before the dump finishes. That’s the boundary where you should stop using Lambda and move to a managed service like AWS Backup or a dedicated instance. It’s a limitation, sure, but I’ve found it covers about 90 percent of the projects I work on.

Why keep it simple?

You get free monitoring via CloudWatch. You pay pennies per execution. You aren’t maintaining a long-running instance just to run a script once a day. (Honestly, I really dislike managing servers when I don’t have to.)

Is there a better way to handle database state for your specific setup? If your data grows, consider how you might switch to snapshotting at the storage level rather than relying on application-level dumps. It’s usually faster and less prone to locking your production tables.