We audited Forge's own pipeline. Here's what we found.

Forge breaks a request into planning, coding, and review steps — a bounded tool loop that reads your files and checks its own changes before anything is applied. We wrote that description ourselves, so we decided to actually check it against the code, function by function, instead of trusting our own summary.

What we checked

Four questions, each answerable by reading the actual code, not guessing from the architecture diagram:

1. Does the sandbox that can safely run generated code actually get called before a change is applied?
2. When the AI adds a new dependency, does anything update the project’s dependency file?
3. Does the review step have the same tools the coding step has — can it read files, not just judge a wall of text?
4. Does the planning step know the shape of your project before it splits work into tasks?

What we found

All four gaps were real. Not “maybe a problem” — verified against the actual function calls.

The review step reads text, not files. The coding step gets five tools (search, read, check its own edit against the real file, etc.). The review step gets one prompt and no way to look anything up. It’s judging code the same way you’d judge a stranger’s pull request through a small window with no scroll bar.

New dependencies go unnoticed. If a fix pulls in a library your project didn’t have, nothing updates requirements.txt. The code can be correct and the project can still be broken the next time someone tries to run it.

The planner splits work blind. It hands out sub-tasks without ever seeing a map of the project it’s working on. A repo_map module already exists in the codebase and is used elsewhere — it just was never wired into the one place that would benefit most from it.

The sandbox exists but sits outside the loop. This one needs a caveat, because the honest version is more nuanced than “unchecked code gets written to disk.” It doesn’t: every change already goes through a syntax check, a cross-file consistency check, a full diff review you see before anything is applied, and an automatic backup. What’s genuinely missing is the AI actually running the code — the sandbox module for that exists and is tested, it’s just never called from the write path.

What we’re doing about it

Nothing shipped yet, on purpose. Before writing a line of the fix, we asked two different AI models — independently, without showing either one the other’s answer — how they’d design the missing piece. They landed on almost the same architecture: run the code in the sandbox first, auto-retry once on a real error with the actual stack trace, then show a human the diff before anything real gets written.

Two independent opinions converging on the same shape is a better signal than either one alone. It’s also not a green light to build it tonight — there are real open questions first (which file in a multi-file change actually gets run, how a sandbox gets the same dependencies your project has, what happens if a test touches a database instead of just the filesystem). Those get answered before code gets written, not after.

The takeaway

We’d rather publish “here’s what’s actually still missing” than a diagram that implies more polish than the code has. If a claim about how Forge works can’t survive us reading the function it describes, we’d rather fix the claim than hope nobody checks.