AI
-
How Do You Actually Review Code an Agent Wrote?
We’ve all had the magic moment by now. You fire up an agent in Claude Code, hand it a bunch of tasks, then let it do its thing. It feels like the future.
Right up until you have to hit
git commit.That’s when the magic curdles into a very specific kind of anxiety: how do I actually review this? I didn’t write it. I barely watched it happen. And now I’m supposed to vouch for it.
The AI Reviewer Trap
The obvious move is to fight fire with fire. An AI wrote it, so pipe the diff into an AI PR review tool and let the machines sort it out, right?
I wrote about this back in May: AI code reviewers won’t save you. Having one LLM grade another LLM’s homework can potentially lead to disaster. They share the same blind spots. They’re trained on the same patterns, so they tend to nod along at the same plausible-looking mistakes, and they’re notoriously bad at catching the subtle, systemic logic flaws that span an entire architecture. The bug that matters is rarely on one line. It’s the interaction between four files, and that’s exactly the kind of thing a second LLM waves through.
So you swing the other way and read it yourself, line by line. Every variable assignment, every branch. And that’s exhausting. Worse, it defeats the entire point of using an agent. If I have to mentally re-type every line the agent produced, I might as well have typed it for real.
So we’re stuck between a reviewer we can’t trust and a review process that erases the speedup. Neither one is the answer.
The Bottleneck Just Moved
Agentic development didn’t make software engineering easier. It moved the hard part.
Writing the implementation used to be the bottleneck. That’s the part the agent is genuinely good at now. What it can’t do for you is tell you the behavior is correct. Verifying the behavior is the whole job now, and that’s a different skill than writing code.
This is why migrating my test suite to Vitest earlier this year has paid off more than I expected at the time. When you’re driving autonomous agents, automated testing stops being a chore you do to keep a coverage badge green. It becomes the only safety net you actually have.
Trust the Spec, Not the Code
In an agentic workflow, my job as the human isn’t to write the function anymore. It’s to write the tests, or at least to rigorously verify the ones the agent proposes.
Think about what that buys you. If I have a comprehensive, fast test suite, I don’t ALWAYS need to read all 400 lines Claude Code just generated. I need to watch the runner light up green. If the tests pass, and the tests are good, the implementation details matter a lot less than they used to. The tests are the contract. The code is just one way to satisfy it.
That second condition is doing a lot of work, though, so I want to be honest about it. “If the tests are good” is the entire game. A passing suite that doesn’t cover the edge cases is worse than no suite, because it hands you false confidence at the exact moment you’ve stopped reading the code. So the scrutiny doesn’t disappear. It relocates. Instead of reviewing the implementation, you review the spec. Are the right behaviors tested? Are the failure modes tested? Did the agent quietly write a test that asserts its own bug?
That’s a much smaller surface to review than 400 lines of implementation, and it’s a far more durable thing to spend your attention on. The tests outlive any single refactor.
TDD Didn’t Die, It Got Promoted
For years people treated test-driven development as a discipline you adopted if you were virtuous and skipped if you were busy. Agentic coding flipped that. It made testing the load-bearing skill, because tests are now the interface between what you want and what the machine builds.
So the answer to “how do I review code an agent wrote” turns out to be: mostly, you don’t. You review what it’s supposed to do, you encode that in tests you trust, and you let the green checkmark tell you whether the agent got there.
I’m curious whether this matches your experience. I’ve found I write more tests now than I did before the agents showed up, not fewer. The implementation got cheap. Being sure it works did not.
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
-
The Agentic Dev Space Is Moving Fast, and I'm Having a Blast
Step back and look at the last six months of software development. It’s almost hard to believe how much has shifted. Back in January we were doing “vibe coding” with basic autocomplete. By June we’re handing fully autonomous agents a feature, walking off to grab a coffee, and coming back to find it architected, written, and tested.
It’s easy to look at the volume of new tools dropping every single day and feel a little fatigued. But honestly? I’m just excited. This is one of the most fun eras of programming I’ve worked through in decades.
Here’s a quick look at the tools that have already changed my workflow, and the ones I want to dig into next.
The Tools I’ve Adopted
If you read my post earlier this week, you know Supacode has become my daily driver. It bridged the gap between raw terminal access and a UI that understands how context-heavy agentic work is. That’s a harder problem than it sounds, and it’s really feeling good so far.
I also have to shout out the tools that paved the way earlier this year. Claude Code and Antigravity proved out the model of a CLI-native agent that could navigate your file system and do the work. Running Claude Code daily—and turning to Antigravity occasionally when I have access—has completely retrained my habits. I stopped typing every line of code and started acting more like a technical lead reviewing pull requests from a tireless junior developer. That shift in posture is the real unlock, and these were the tools that taught me it.
The Tools on My Radar
Because the space moves so fast, my “stuff I want to learn” backlog keeps growing faster than I can clear it. Here are the three I’m most eager to explore when I carve out the time:
- Hermes. I keep hearing great things about how it handles multi-step reasoning and huge context windows. I haven’t had the right project to throw at it yet, but it’s at the top of the list.
- Pi Agents. The concept here is highly specialized, networked agents working in tandem. Instead of one monolithic agent doing everything, you’d have a “frontend agent” talking to a “database agent.” That feels like a different way to structure the work, and I want to see if it holds up in practice.
- Evaluating Memory Systems. I’ve been diving into how agents remember things over time. I wrote a Mem0 MCP server that runs locally, and I’ve added hooks for Mem0 right into my CLI agents. It’s been a great way to slowly improve my own local agentic memory system. I’m really eager to see how other memory systems (like LangChain’s implementations) work under the hood, and I want to spend more time comparing and contrasting them.
You Don’t Have to Learn It All Today
The best part about this explosion of tooling is that you don’t need to know all of it. You don’t.
Find a tool that solves an immediate problem in your workflow, master that one, and let the rest sit on your radar until you need them. The ecosystem will keep evolving whether or not you’re watching every release. The tools will keep getting better. Chasing every launch is a great way to learn nothing well.
So pick one. Get good at it. Let the backlog wait.
What’s the one agentic tool sitting on your radar that you just haven’t had time to dig into yet?
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
-
Why Supacode Became My Daily Driver for Claude Code
The number of ways to get an AI to write your code right now is … great? Claude Code, Codex, Opencode, and all the others. I think that’s a good thing. The easier we make it to hand real work to a capable coding agent, the better off we all are in the long run. I want to try most of them because that’s half the fun, figuring things out.
My cool tool for the week is Supacode.
What Supacode Is
Supacode is a native macOS app, written in Swift and built on libghostty, with no Electron and no web wrapper anywhere in sight. It’s fast the way native software is fast, the kind of speed you feel in every keystroke.
The website calls it a “command center for coding agents,” and well ok I get it now. Supacode is the nicest way I’ve found to run Claude Code. It’s open source too, so you can poke at the internals on GitHub if you’re curious.
What It Does for Me
This is the part that I like.
Supacode keeps all my projects organized down the left side, so jumping between them is one click instead of a mental map of which terminal tab is which. When I start a Claude Code session, its chat window moves up into the active area, and I can see every live session I have going at a glance.
The big one: it hooks into Claude Code, so Claude tells Supacode when it’s finished a task or when it needs something from me. Instead of babysitting a terminal waiting for the next prompt, I just get told when Claude is waiting on me. I tried for a long time to get something like that working in a plain terminal and never really nailed it. Here it just works.
And when I need more than one CLI open on the same project, I can group those tabs together instead of squinting at a wall of identical-looking shells.
None of that is glamorous. It’s just the difference between running Claude Code and pleasantly running Claude Code.
I Still Love Ghostty
I’m not leaving Ghostty behind, to be clear. I love Ghostty. Mitchell Hashimoto and everyone who’s worked on it has built something special. When I need a clean shell to compile a binary or grep through some logs, Ghostty is where I want to live.
Supacode is built on libghostty, the same engine that powers Ghostty. So the speed and feel I love about Ghostty is sitting right underneath Supacode too. They’re cousins. I get Ghostty for raw terminal work and Supacode for running Claude Code, and both of them are quietly standing on the same excellent foundation.
So Here We Are
We’re spoiled for choice with AI coding tools right now, and I love that. Most of what I try is interesting, and then I move on. Every so often something just slots into how I already work and earns a permanent spot. For me, right now, that’s Supacode. If you’re running Claude Code all day on a Mac, it’s well worth a look.
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
-
Adaptive | AI agents that run your business
Sell, market, follow up, research, and build — all from your desk or your pocket.
-
A Dotfiles Manager That Snapshots Every Change
Managing dotfiles in 2026 is a solved problem in the same way that managing your own backups is a solved problem: there are five tools for it, all of them work, all of them require you to set up some plumbing first, and once you’re set up you still don’t have a great answer to “I just broke my shell config, get me back to yesterday.”
The conventional answer is some combination of: a git repo for your
~/.zshrcand friends, a symlink script (orstow, orchezmoi, oryadm), and the discipline to remember to commit after every change. The setup is a one-time hassle. The “wait, what did I change?” recovery story is not great. And if you want to sync across machines, you’ve now got opinions about remote repos, SSH keys on a fresh box, and which order things have to happen in.I wanted something different, so not a configuration framework, but a record of every change to the files I care about, in a place I can roll back from, with the lowest possible setup cost.
That’s what dfm is.
What It Does
dfmis a single static Go binary. You point it at the files you want to track (~/.zshrc, anything under~/.config/, whatever), and every time one of them changes it takes a content-addressed snapshot. The snapshots live on disk in~/.local/share/dotfiles/backups/. A small state database (SQLite locally, or libSQL via Turso if you want cross-machine sync) records which file maps to which snapshot at which point in time.You can roll back. You can diff against an old snapshot. You can see when you last touched a file. And because every snapshot is content-addressed, you never re-store the same bytes twice — switching themes in
~/.zshrcten times costs the size of two configs, not ten.The other half is the backup story.
dfm initwalks you through cloning (or creating, viagh) a private GitHub repo that mirrors your tracked files plus their history. The point isn’t to make you adopt a new git workflow. It’s that pulling your config onto a fresh machine should be one command, and recovering fromrm -rfshould never have a “well, hopefully my last commit was recent” caveat.Why Setup Is the Hard Part
The reason people don’t audit their dotfiles is the same reason people don’t back up their laptops: the setup is annoying, and the payoff is theoretical until it isn’t.
dfm initis a six-step interactive wizard. It detects aTURSO_DATABASE_URLenv var if you’ve got one, offers sensible defaults for everything else, lets you opt in to tracking~/.zshrcimmediately, and writes a single config file with the right permissions. Re-run it on an existing config and it pre-fills every prompt with your current value, so the cost of changing your mind later is also low.--yesaccepts every default for scripted setup.If that sounds boring, that’s the point. Boring is what makes a tool actually get used.
The AI Bit
There’s an optional AI integration.
dfm suggest <file>asks a local AI CLI (Claude Code by default, configurable) to propose an improvement to one of your tracked files, returns the proposal as a unified diff, and stores it as a pending suggestion.dfm apply <id>reviews the diff and applies it, with a fresh snapshot first, so you can roll back if the suggestion turns out to be wrong.I’m exited to try this feature out, because I’m sure there is something i"m doing wrong. The “Look at my
~/.zshrcand tell me what I could clean up” is useful feature that doesn’t require me copy and pasting or granting read or write access to my entire home directory.Where to Get It
github.com/llbbl/dotfiles-manager. Pre-built binaries for darwin and linux on arm64/amd64. Current version, as of writing, is v1.4.0.
If you’ve been meaning to actually back up your dotfiles and the friction has stopped you, this is the post where I tell you the friction is solvable.
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
-
Your AI Coding Agent Can Read Every Secret on Your Machine
Every developer running an AI coding agent has handed that agent the keys to their machine. Not metaphorically. Literally. The agent runs as your user. It can read every file you can read, execute every command you can execute, and hit every API your stored credentials authorize.
For most workflows, that’s the point. You want the agent to read your code, modify your project, ship your work. But there’s a quieter implication: the agent can also read your
.envfiles. It can invoke your secret-management tooling. It can grep forAPI_KEY=across your home directory. And nothing in the agent stack says “wait, you didn’t ask for this.”Same-UID isolation isn’t isolation. It’s the absence of isolation labeled politely.
The usual answer to “keep secrets safe from your coding agent” is: don’t store them where the agent can find them. Use a cloud secret manager. Rotate aggressively. These are good practices, and for local development, they’re often impractical. The agent is going to encounter secrets whether or not your security-best-practices doc approves.
So over the last week, I built an audit subsystem into lsm, my Local Secrets Manager. The whole thing is designed to answer one forensic question: did anything weird touch my secrets last night?
The Threat Model
A defense without a threat model is theater, so let me be specific.
The threat isn’t a sophisticated remote attacker. lsm is public, open-source code. The threat isn’t a buggy lsm either; bugs happen, and the user can read the source.
The threat is the agent layer running adjacent to lsm. Coding agents have legitimate access to a wide swath of your filesystem. They’re imperfect at intent inference. They sometimes get prompt-injected. They sometimes run in the background while you’re asleep. When an agent calls
lsm get prod DATABASE_URL, the action is indistinguishable from you doing the same thing. The audit log’s job is to make those calls retrospectively distinguishable.A secondary threat is an agent covering its tracks. If something reads a secret and then edits the audit log to erase the evidence, the log is worse than useless.
What Got Built
The audit subsystem records every access as a structured event: a sequence number, a timestamp, the action, the app and environment, an
Actorblock describing the calling process, and two cryptographic fields linking each event to the previous one.The
Actorblock was the interesting design problem. It captures parent process ID, parent process name, TTY device path (or empty if there’s no terminal), current working directory, an agent marker derived from environment variables that tools like Claude Code, Cursor, Aider, and Continue set, and the calling user ID. Every field is captured every time. Noomitempty. UID zero is a real, meaningful value, and silently dropping it would be a footgun.Events land in a hash-chained JSONL file at
~/.lsm/audit.jsonl. Each row carries the SHA-256 of the previous row plus its own body. If anyone edits, inserts, or deletes a row in the middle, the next row’sprevno longer matches andlsm audit verifysurfaces the break.The chain doesn’t catch tail truncation. If you chop off the end of the file, what’s left is internally consistent. A sidecar file storing the last expected hash is the obvious fix, and I deliberately rejected it. lsm is public code. Any local attacker who knows about the sidecar can rewrite both files in lockstep. Tail-truncation detection is deferred to the off-machine path: when events ship to a remote stack, the last hash naturally lives somewhere the local attacker doesn’t control.
Reading the Log
Three commands cover the read side.
lsm audit taildoes what you’d expect.lsm audit show <seq>prints a single event.lsm audit queryis the workhorse, with every field as a filterable dimension:--app,--env,--event,--parent-comm,--agent-marker,--tty present|absent,--since,--until. Output is JSONL when piped and columnar text when interactive.Then there’s
lsm audit suspicious, which runs four hard-coded detectors in one pass:- Outside hours. Events whose timestamps fall outside 07:00–23:00. The 3 a.m. canary.
- Burst. More than N events from a single parent process within a sliding window. The runaway-agent canary.
- New parent_comm. Process names not seen in the prior 30 days. The “what is this new thing” canary.
- Non-interactive, no agent. No TTY, no recognized agent marker. The “what is even running this” canary.
A single event can stack reasons. A 3 a.m. burst from an unknown parent is unambiguously interesting.
The detector doesn’t learn baselines, doesn’t call out to an ML model, doesn’t require a service. High-signal patterns are obvious patterns, and obvious patterns are well-served by hard-coded predicates.
Shipping Events Off the Box
If you already run an observability stack, lsm can ship audit events over OTLP (the OpenTelemetry wire protocol). Three design choices matter here.
The local file sink is always authoritative. The remote sink is a mirror, not a replacement. An lsm operation never fails because the remote endpoint is down.
Redaction is allowlist-based. App and environment names are HMAC-hashed with a per-host salt before becoming labels. The TTY device path is dropped and replaced with a
tty_present: true/falseboolean. Secret values,cwd,hash,prev, and the schema version never leave the host. Secret names are replaced withkey_present: truemarkers; the remote observer can see that a key was accessed, never which key.Events whose name starts with
audit.(chain failures, suspicious matches, sink drops) are always local. Telling a remote attacker that local integrity has been compromised is counterproductive.What’s Still Open
The most important non-feature: no command in lsm emits events yet.
setdoesn’t log.getdoesn’t log.deletedoesn’t log. The plumbing is complete, the calls are not wired in. Each emit site needs careful thought about which fields are appropriate, whether the event should be local-only, and how it interacts with sensitive operations. That’s the next chunk of work.The agent-coding era is normalizing a model where AI tools have wide-ranging access to developer machines. The premise that the agent operates as a fully-trusted local user is unlikely to change soon. Managing the risk means visibility. It means being able to answer “what touched my secrets last night” with a record the agent couldn’t silently rewrite.
The code is at github.com/llbbl/lsm. The full design lives in
docs/observability.md.I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
-
Sandboxing AI Agents Without Buying Anything
The previous post (and the one before it) covered the threat model and the per-ecosystem mitigations: lockfiles,
--ignore-scripts,cargo-audit, Trusted Publishing. All of that helps. None of it answers the question that keeps me up at night, which is: what happens when an AI agent on my laptop installs a malicious package, and the malicious package was the literal point of the operation?This is the new shape of the threat. You’re not getting compromised because you typed
npm installwrong. You’re getting compromised because Claude or Cursor confidently invented a package name that didn’t exist, an attacker registered it five hours ago, and the agent ranpip install hallucinated-thingon your behalf without asking. The agent has shell access. Your SSH keys are right there. Your~/.aws/credentialsfile is right there. The entire premise of giving an AI agent the ability to just figure it out depends on it being able to execute untrusted code at the speed of conversation, which is also the worst possible threat model.If you’re a solo developer, an open-source maintainer, or a startup with no budget for Socket or Endor Labs licenses (more on those next post), the answer isn’t a commercial firewall. The answer is local isolation, and the tools have gotten dramatically better in the last 18 months.
Containers as the Baseline
The minimum viable isolation in 2026 is don’t run untrusted code as your user on your host OS. The cleanest way to do that on macOS or Linux is a devcontainer, a fully described, reproducible Linux environment that VS Code, Cursor, and the Claude Code CLI all natively support. You give the agent the container as its sandbox. Project files mount in. SSH keys, AWS credentials, and the rest of your home directory don’t.
The container runtime matters. Docker Desktop on macOS is a memory pig, 3 to 4 GB resident at idle, with sluggish startup times that make iterative work miserable. OrbStack is the obvious replacement: free for personal use, native Apple Silicon, dynamically allocates memory instead of reserving fixed blocks, and benchmarks show container startup times around 0.2 seconds versus Docker Desktop’s multi-second cold starts. If Docker Desktop is eating half your RAM before you even start Claude Code, OrbStack will give you that memory back.
The thing to internalize, though, is that a container is not a security boundary by default. It’s a deployment mechanism that happens to have isolation properties when configured correctly. Misconfigured developer containers have been implicated in some of the largest crypto-industry breaches of the last few years. The pattern: a container running with privileged flags, or mounting the wrong host directory, turns into a path straight to the host. Containers help. They don’t save you from yourself.
The configuration mistakes that void the isolation:
- Mounting
~/.sshinto the container so the agent cangit push. Now any process inside the container can read your SSH keys. - Mounting your entire home directory as a convenience. Now everything is accessible.
- Running with
--privilegedor sharing the host’s Docker socket. Container escape becomes trivial. - Letting the agent run
sudoinside the container. The container’s root can chain to host kernel exploits.
Least privilege, applied seriously. The agent gets the project directory and nothing else. If it needs to commit, it pushes through a credential helper that lives on the host, not by mounting your SSH keys.
Lighter-Weight Sandboxes
Spinning up a full container for every test this snippet the LLM wrote interaction is too heavy. There’s a middle layer worth knowing about.
Python. Pyodide compiles CPython to WebAssembly, which means Python code runs in a deny-by-default memory sandbox with no filesystem or network access unless you explicitly grant it. Works great for evaluating LLM-generated snippets, struggles with C extensions and heavy dependencies. safe-py-runner is the pragmatic alternative: it runs Python in a restricted subprocess with timeouts, memory limits, and I/O marshaling. No container needed. For code that absolutely cannot touch your machine, remote V8-isolate services like Deno Sandbox boot pre-snapshotted Python environments in the cloud and air-gap execution entirely.
Rust. The
build.rsproblem from the last post has no first-class solution yet, but on Linux you can wrapcargo buildin Landlock, a kernel feature available on 5.13+ that lets unprivileged processes restrict their own filesystem access. Combined with seccomp-bpf for syscall filtering and cgroups v2 for resource limits, you can run a build script that genuinely cannot read your SSH keys or open arbitrary network sockets. Projects like sandbox-rs wrap these primitives into something usable without writing your own seccomp filters. None of this works on macOS without a Linux VM in the way, which is another reason OrbStack plus a devcontainer is the path of least resistance for most people.The Mindset Shift
The honest version of all of this: if you’re running AI agents locally, you have to assume they will eventually install something malicious. Not might. Will. The question is whether the blast radius is the contents of one project directory inside a container, or every credential on your machine plus your entire git history. That gap is what isolation buys you.
Containers, Landlock, WASM sandboxes, none of these are particularly hard to set up. They’re just things most developers haven’t bothered with because the threat model didn’t feel real. After Shai-Hulud, faster_log, and a year of watching AI agents
pip installwhatever they invent, the threat model is real.Next post I’ll wrap up the series with the commercial side: Socket, Snyk, Endor Labs, Mend, Sonatype, the pricing comparison, and the actual ROI math for whether any of it makes sense for teams below 50 developers.
Sources
- State of Dependency Management 2025 — Endor Labs
- Securing AI Coding Assistants: A Total Cost Analysis — Endor Labs
- A step closer to isolation — devcontainer-wizard — The Red Guild
- OrbStack vs Docker Desktop: Performance Facts for Mac
- Apple Containers vs Docker Desktop vs OrbStack benchmark
- How to Safely Run AI Agents Like Cursor and Claude Code Inside a DevContainer
- DevContainers for Secure AI: Isolated & Scalable
- safe-py-runner: Secure Python execution for LLM Agents
- mcp-run-python — Pydantic
- How to Run Rust Binaries Without Root Using Sandboxing — OneUptime
- sandbox-rs
- Explore sandboxed build scripts — Rust Project Goals
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
- Mounting
-
SAST vs AI PR Review: Two Tools, Different Jobs
If you have worked in DevSecOps, you might be wondering if AI pull request review tools are going to replace traditional SAST scanners. Short answer: no. Longer answer: they’re solving different problems, and if you’re picking one over the other, you might be making a mistake.
Here is how I think about it.
SAST is the Compliance Gatekeeper
Static Application Security Testing tools, think Semgrep, SonarQube, Checkmarx, Fortify, parse your source code (usually into an Abstract Syntax Tree) and hunt for known vulnerability patterns. They don’t run the code. They just read it and “pattern-match” against rules.
The focus here is security, compliance, and strict rule enforcement. SAST is the automated gatekeeper that makes sure your code clears the OWASP Top 10 bar before it merges.
What SAST does well:
- It’s deterministic. If a rule matches a pattern, the engine flags it every single time. Run it twice on the same code, get the same result.
- It satisfies auditors. Frameworks like PCI-DSS, SOC 2, and HIPAA expect documented secure-development practices, and a formal SAST scanner is the easiest way to produce that evidence. AI agents don’t count here, at least not yet.
- It can do real taint analysis. Enterprise tools can track untrusted input from the moment it enters your app to the moment it hits a dangerous sink.
Where SAST falls down:
- The false positive rate is brutal. Rigid rules with no context means a lot of noise. Developer fatigue is real, and once your team starts ignoring scanner output, you’ve lost the game.
- It can’t see your business logic. A SAST tool has no idea what your application is supposed to do, so it can’t tell you when the logic itself is broken.
- Comprehensive scans are slow. Hours on large codebases isn’t unusual, though Semgrep has been doing good work on this front.
AI PR Agents are the Peer Reviewer
Tools like CodeRabbit, Qodo, Greptile, GitHub Copilot Code Review, Cursor Bugbot, and Claude Code (set up as a review skill) plug into your version control and read the PR diff with the surrounding code context. They behave less like a scanner and more like a colleague who actually read your changes.
The focus is developer productivity, code quality, logic bugs, and contextual feedback.
What they do well:
- They understand intent. LLMs can reason about why the code is changing, not just whether it matches a rule. That’s a different category of feedback.
- The signal-to-noise ratio is good. When an AI flags something, it usually comes with an explanation that makes sense. Less noise, more useful comments.
- They suggest fixes. Not just “this is wrong” but “here’s a diff you can apply.” That’s huge for actually closing the loop on review feedback.
- The scope is broader. Architecture, performance, style, security, all in one pass.
Where they fall down:
- They’re non-deterministic. Same vulnerability, two PRs, two different outcomes. That’s not a bug, that’s how LLMs work, and it’s why auditors don’t trust them.
- They don’t satisfy compliance. No auditor is going to accept “the AI looked at it” as a substitute for a formal scanner.
- Hallucinations happen. Invented issues, misread intent, suggestions that refactor things that didn’t need refactoring. You still need a human filtering the output.
The Quick Comparison
Feature SAST AI PR Review Primary Goal Security & Compliance Code Quality & Productivity Analysis Method Deterministic rules & AST Non-deterministic LLMs Business Logic Blind Context-aware False Positives Often high Usually low Compliance Proof Accepted as evidence Not accepted Feedback Loop Dashboard / CI output PR comments / chat The Lines Are Starting to Blur
The interesting thing happening right now is convergence from both directions.
On the SAST side, tools like DryRun Security are pitching themselves as “AI-native SAST,” trying to keep the deterministic backbone while using LLMs to filter out the false positives that make traditional scanners painful to live with.
On the AI agent side, CodeRabbit and Greptile keep getting better at catching real security vulnerabilities, not just style issues. They’re slowly creeping into territory that used to belong exclusively to SAST.
This is going somewhere, but it’s not there yet.
Where to Start Your Evaluation
Treat them as complementary, not competitive.
For SAST, evaluate against your audit footprint, the languages in your codebase, and how much false-positive triage your team can absorb. Semgrep, SonarQube, Checkmarx, and Fortify all sit in different price-and-friction zones, and the right one depends on what your business actually needs to prove.
For AI PR review, evaluate based on how it fits your existing review workflow, what languages and frameworks it understands well, and the signal-to-noise ratio in practice on your codebase. CodeRabbit, Qodo, Greptile, Copilot Code Review, Bugbot, and a Claude Code review skill all approach the problem differently.
If you pick one category and skip the other, you’re either passing compliance with mediocre code review, or getting great review feedback while failing your next audit. Neither is a win.
The AI tools aren’t replacing SAST. They’re filling in the gap SAST was never designed to cover.
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
-
AI Code Reviewers Won't Save You
Dropping an AI reviewer into your pull request pipeline is just a band-aid. Tools like CodeRabbit or Greptile are great for catching syntax errors or basic anti-patterns, but they can’t assess architectural intent or domain-specific business logic. They’re spell-checkers for code. Useful, sure. But nobody ever said “our codebase is solid because we run spell check.”
AI doesn’t change your engineering baseline. It just accelerates it. If your foundational guardrails are weak, agentic tools will help your team generate technical debt at unprecedented speeds. So the real question isn’t “how do we review AI code?” It’s “how do we build systems that prevent slop from ever reaching production?”
Shift Left, Hard
When engineers use agents to scaffold a new Go service or spin up a SvelteKit frontend, they’re inevitably pulling in generated dependencies or utilizing unfamiliar libraries. Models hallucinate packages. They suggest insecure patterns with total confidence.
Your CI pipeline needs to be ruthless before a human ever looks at the code. Aggressive SAST and SCA should automatically block PRs that introduce vulnerable dependencies or hardcoded secrets. If the agent generates slop, the pipeline rejects it instantly. No discussion.
Make the Agents Write the Tests
Agents are incredibly eager to generate feature code, but humans are historically lazy about writing the tests for it. The influx of AI-generated code means human reviewers can’t possibly step through every logic branch manually.
So flip the script. Use the agentic tools to build the guardrails themselves. Mandate that any generated feature code must be accompanied by generated, human-verified unit tests. If an agent writes a sprawling TypeScript function, the build should fail if the test coverage doesn’t meet a strict threshold. You’re already using AI to write the code. Use it to prove the code works, too.
Context Boundaries Matter
Bloated AI output often happens because the model is given too much context or allowed to generate too much at once. Heavyweight IDEs with aggressive multi-file auto-completion can easily create cascading messes across a codebase.
Define strict architectural boundaries and API contracts upfront. Agents should be tasked with solving small, well-defined, modular problems. “Write a function that parses this specific JSON schema” is a good prompt. “Build the backend” is not. The tighter the scope, the less room for generated nonsense.
Observability Is Your Safety Net
You can’t catch all generated slop at the PR level. Some of it only reveals itself under load. An agent might write a technically correct query that causes an N+1 database issue, or introduce a subtle memory leak that passes all unit tests.
Your ultimate safety net is what happens at runtime. You need an airtight observability stack to trust the velocity AI brings. Logs, distributed tracing, metrics, all feeding into dashboards your team actually watches. When generated code hits staging, you need the immediate telemetry to spot performance regressions before they reach production.
Redefine the Human Review
Because AI makes the “typing” part of coding trivial, the human code review needs to fundamentally shift. Reviewers should no longer be looking for missing semicolons. They should be asking: “Does this component fit our architecture?” and “Did the agent over-engineer this solution?”
Train your senior engineers to review for intent and systemic impact. That’s the stuff AI genuinely can’t do yet. Leave the syntax checking to the robots.
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].
-
Leading Teams When AI Does the Typing
AI is changing how teams build things. That part is obvious. What’s less obvious is that it doesn’t change what makes people want to follow you.
If anything, as the technical execution gets more automated, the human stuff becomes more important. Empathy, vision, strategic judgment. This has been top of mind for me a lot lately. I think the leaders who are going to thrive aren’t the ones who adopt AI the fastest. They’re the ones who understand what AI can’t do.
Focus on Outcomes, Not Output
AI accelerates generation. Code, docs, data parsing. All of it gets faster. That means individual output spikes, and measuring your team’s success by raw productivity becomes a losing game.
Your job is to provide the why. AI can handle the how, but it cannot determine why you’re building something in the first place. Crystal-clear business context and architectural vision are what keep your team’s AI-augmented velocity pointed in the right direction.
Something worth thinking about: when engineers can generate solutions faster, system complexity increases. Fast. Leadership means making sure the team is building the right things, not just building things quickly. Speed without direction is just expensive chaos.
Build the Guardrails Before You Need Them
Before you hand a team tools that let them move at lightspeed, the brakes and steering are important to have in place.
Automate your compliance. Strong SAST/SCA tooling, foolproof secret management, the boring stuff that lets people experiment without risking the infrastructure. As AI assists with more logic and agents take on more autonomous tasks, these guardrails become non-negotiable.
Same goes for observability. You can’t manage what you can’t see. When AI agents are handling high-volume tasks, a solid observability stack is what lets you trust the automation and catch it when a model hallucinates or a system drifts.
Automate the Toil, Protect the Thinking
A good leader uses AI to elevate human effort, not replace it. Look for the most repetitive, low-joy tasks in your team’s workflow. Scaffolding test environments, parsing logs, summarizing meetings. Deploy AI to handle the toil so your people can do the work that actually requires a brain.
Then protect that time. With AI handling the busywork, guard your team’s calendar for deep architectural thinking, complex debugging, creative system design. The stuff AI still struggles with. That’s where your team’s real value lives.
The Human Skills Are the Whole Game Now
The most valuable skills in an AI-driven org are the ones algorithms can’t replicate.
- Psychological safety. AI tools are changing fast. Your team needs to feel safe experimenting, failing, and learning. Punish well-intentioned experimentation and you’ll kill innovation before it starts.
- Mentorship. AI can answer factual questions all day long. It cannot mentor a junior engineer through a crisis of confidence or help a senior navigate organizational politics. Put your energy into 1:1s, career mapping, and active listening.
- Ethical judgment. As AI agents take on more responsibility in domains like finance, underwriting, or automated operations, you are the moral compass. You’re the one who needs to ask the hard questions about bias, fairness, and unintended consequences.
So What’s the Job Now?
The job is the same as it’s always been: create clarity, remove obstacles, and give a damn about your people. AI just raises the stakes on all three.
The leaders who get this right won’t be the ones with the best AI strategy deck. They’ll be the ones whose teams actually want to show up and build something together.
I’d appreciate a follow. You can subscribe with your email below. The emails go out once a week, or you can find me on Mastodon at @[email protected].