Automating Code Reviews With AI

I get why people hate AI in code reviews. It often hallucinates, suggests broken syntax, or just adds noise to an already busy pull request. Honestly, sometimes I think I’d rather just ignore it entirely. But, if you configure it right, it actually helps.

Why my first attempt failed

I once piped every diff directly into a generic chatbot prompt. The result? A disaster. It flagged perfectly valid logic as a performance bug just because I used a nested loop. I was confused, annoyed, and eventually turned it off. It wasn’t smart enough to know my business context, and it lacked any sense of project architecture.

Then I realized my mistake. I was treating the AI like a senior engineer. It isn’t one. It’s a glorified linter with an ego. Once I scoped it down to specific, narrow tasks, things changed. Here is how I set up a basic check for security leaks using a custom prompt in my CI pipeline:

1
# A simple script for PR comments
2
if [[ -n $(git diff --name-only | grep ".ts") ]]; then
3
  cat diff.patch | llm-cli --prompt "Check for hardcoded secrets."
4
fi

Does this actually scale?

“But Namir, my team reviews everything manually, and we don’t need a bot telling us about minor linting issues.” You’re right. If your team is small and everyone is senior, you don’t need a bot doing the heavy lifting for simple syntax. But manual reviews fail when you get tired or rushed. That’s where the machine wins.

I admit, it struggles with complex refactoring. If you ask an AI to rewrite a massive legacy module, you will end up with broken code. I wouldn’t trust it with critical business logic (maybe because I don’t trust anyone else with it either). It’s best when used for the boring, repetitive parts of the review process.

Keep the human in the loop

Don’t let the AI merge anything. Ever. I set my tool to only post comments or suggest changes that a human must manually accept or dismiss. This keeps the authority where it belongs. (It’s really just a specialized CI step, not a replacement for a brain.)

Start by offloading the stuff you hate reviewing, like variable naming consistency or checking if environment variables are properly used. If you give it a clear, constrained task, you might find that it saves you ten minutes of repetitive clicking per PR. Is it perfect? No. Is it useful? Absolutely.

What’s the most frustrating thing you’ve seen an AI suggest in a code review?