Context Engineering Isn’t Enough — A Loop Engineering Experiment With No LLM Inside the Loop
The Scope: This isn’t a benchmark pitting agent frameworks against each other, nor does it claim this controller is “smarter.” It claims something narrower and much more defensible: that failure isolation is a real, measurable architectural property.
It’s a Pattern, Not a Product: Loop engineering is a control-flow pattern, not a specific framework. It’s the concept of replacing a single giant prompt with an iterative system that observes state and acts toward a goal.
Built for Verification: To prove this mechanism rather than take it on faith, I built a tiny, deterministic implementation of a goal-directed controller. It requires zero LLM calls and has zero external dependencies.
Measurable Failure Isolation: While a linear, one-shot pipeline halts entirely on its first unresolved obstacle, this controller isolates failures to the specific branch that contains them. Across 300 random seeds, the controller completed a mean of 3.3 out of 10.3 independent branches, compared to just 0.4 for the linear baseline.
Radical Transparency: I actually found and fixed a real bug in my own benchmark logic before trusting these numbers, and I’m walking through that mistake in detail rather than hiding it.
The Pipeline That Stopped for No Reason
A few months ago, I watched a task-processing pipeline collapse on step three of forty.
Step three needed a configuration value that I hadn’t set yet. Because of that single missing value, everything downstream just sat there dead. That included thirty-some steps that had absolutely nothing to do with the missing config.
The work itself wasn’t impossible. The system collapsed simply because the pipeline had no conceptual ability to skip a broken branch and keep moving on the others.
This is not a prompt problem. No amount of rewriting instructions for the model would have fixed it, because the model was never the part that broke. The failure happened in the control code. The logic responsible for deciding what to do next when things went sideways didn’t decide anything. It just quit.
I am writing this to address the layer completely above the prompt. This is the part of an agentic system that manages state and determines the next move after a step succeeds, fails, or gets stuck. In AI engineering circles, people call this loop engineering.
Before you read any further, I want to be entirely precise about what I did and did not do here. Precision is the whole point of this piece.
- I did not build a new agent framework.
- I did not benchmark an LLM.
Instead, I built one small, deterministic, thoroughly tested piece of software to prove a single architectural claim. I am going to show you the code, expose the bugs I had to catch in my own benchmark logic before I trusted the data, and lay out the exact numbers.
I don’t expect you to take this mechanism on faith. I want you to check my work.
What “Loop Engineering” Means, and Where the Term Comes From
If you spend time on AI engineering Twitter, Substack, or LinkedIn, you have probably seen the phrase loop engineering. I want to be precise about its origin rather than treating it as ambient knowledge. Misattributing a fast-moving term is exactly how you erode trust in a technical article.
Google engineer Addy Osmani named and structured the term in an essay published in June 2026. He built on a line from developer Peter Steinberger, who argued that practitioners should stop prompting coding agents directly and instead design the loops that prompt them. Around the same time, Anthropic’s Claude Code lead, Boris Cherny, made a similar point about his own workflow shifting from writing prompts to writing the loops that write prompts. Osmani’s essay frames loop engineering as sitting one level above what he separately calls harness engineering, the layer concerned with the environment a single agent runs inside. Other practitioner write-ups extend this into a fuller stack, prompt engineering to context engineering to harness engineering to loop engineering, with each layer wrapping the one before it.
We should also credit a direct technical predecessor. Engineer Geoffrey Huntley described running a coding agent inside a plain while loop in July 2025, an approach that became known as the Ralph technique. The core idea, feeding an agent the same goal repeatedly until it satisfies a written spec, is a simpler ancestor of the exact same pattern.
Osmani’s essay describes five building blocks for a self-running loop, automations, worktrees, skills, plugins and connectors, and sub-agents, plus external state as a sixth, and multiple explainers cover that architecture in more depth than I will here.
What I am doing in this article is much narrower. Loop engineering is the overarching pattern. What follows is one small, deterministic implementation of a single piece of that pattern, built so you can inspect the mechanism directly rather than take it on faith.
Why I Built This Without Calling a Single LLM
Every explainer I read about loop engineering assumes an LLM sits inside the loop, making the decisions at each turn. That is a reasonable assumption for products like Claude Code, which genuinely put a model at the center of the decision loop.
But this creates a problem if you want to evaluate the architecture on its own terms. If the loop’s behavior depends entirely on what the model decides to do, you can never fully separate a good loop design from a model that just got lucky on a specific run. Nondeterminism is the right tool when you need judgment. It is the wrong tool when you want to verify a claim about control flow.
So I built the exact opposite. I wrote a goal-directed control loop where a plain Python rule handles every decision instead of a model call. It requires no API keys, costs no tokens, and introduces zero variance between runs using the same seed.
If this architecture offers a real advantage over a linear, one-shot pipeline, that advantage should show up even when an if-statement represents the intelligence at each step. If it fails to show up here, dressing the same control flow in an LLM wouldn’t make the underlying claim any more true. It would just make the claim harder to check.
Hold onto this specific distinction for the rest of this piece:
- Loop engineering is the general design philosophy. You replace a single large prompt with a system that observes state, acts, and iterates toward a goal.
- The goal-directed controller I describe below is one concrete, deterministic implementation of that philosophy. I built it to test one specific claim about failure isolation.
Conflating these two concepts turns articles about this topic into unfalsifiable thought pieces. I would rather hand you code that you can actually run.
What I’m Not Comparing, and Why That’s the Honest Scope
Before I break down the architecture, I want to head off a comparison that this article is not making. Anyone who has used LangGraph, CrewAI, or AutoGen will reasonably wonder how my code stacks up against those frameworks.
Let me be clear. I am not benchmarking one agent framework against another. I am not comparing GPT-5 against Claude, and I am not evaluating reasoning quality at all, since nothing in my code actually reasons. Instead, I am isolating a single architectural property. I want to see what happens to the rest of a workflow when one part of it hits an obstacle it cannot immediately resolve.
A linear executor represents the default shape of most one-shot pipelines, and it simply stops when it encounters an error. A goal-directed controller behaves differently. Even a completely deterministic controller with no model inside it reroutes around the parts of the graph it cannot currently move forward, and it keeps working on the parts it can.
That is a narrow, checkable claim, and it is the only claim I am making in this article.
The Architecture: A State Machine, Not a Sequence
The system I built consists of two parts. First, it has an environment, which is just a graph of tasks with dependencies. Second, it has a controller, which is the logic that decides what to do about each task during every iteration. I made the environment intentionally uninteresting. It is a simple directed acyclic graph and nothing more. The controller is the actual subject of this article.
I designed a specific state machine for this setup. Every single task moves through this cycle on every iteration until the entire graph reaches a stable end state:
Every task lands in whichever branch its current state calls for, independently, on every single iteration. This process continues until the entire graph reaches a terminal state where every task is either done, permanently failed, or provably deadlocked.
Nothing about this setup requires a fixed sequence of steps. This is the actual mechanical difference between my implementation and a linear pipeline. It is worth sitting with this concept for a moment because it is the entire source of the benchmark results I show later in this article.
One specific design detail matters far more than it initially appears. When a task needs a resource it does not yet have, the controller must tell the difference between two scenarios. It needs to know if a resource is still resolving and needs a recheck next iteration, or if that resource is never going to resolve, no matter how many times you ask.
I will come back to this point later. Getting this distinction wrong was the exact bug that nearly cost me a valid benchmark.
Where a Real Reasoning Engine Plugs In
To execute a task, the controller calls task.action(task, context). In this implementation, I wrote that call as a plain, deterministic Python function.
That specific call is the seam. If you swap that function for an LLM invocation with the exact same signature, you can pass the task and its context, return a success or failure state, and leave every other line of code in the controller completely unchanged.
The control flow loop of observing, retrieving, asking, executing, revising, or blocking does not know or care whether a hardcoded rule or a frontier model makes the final decision. This clean separation between the control flow and the reasoning engine is the exact architectural argument I am trying to make. The architecture itself is completely model-agnostic.
Building an Environment Worth Testing Against
A control loop is only interesting if you give it real obstacles to navigate. To achieve this, I generated synthetic task graphs and seeded them for strict reproducibility. I structured these graphs as independent branches that feed into a small number of integration tasks at the very end. This setup mimics a real-world production pipeline where several unrelated pieces of work eventually must combine.
I randomly assigned each task one of six distinct behaviors. I used roughly equal odds for these assignments, though I heavily weighted the distribution toward the clean case.
By design, roughly a quarter of the tasks are permanent dead ends. I did that deliberately. I did not want to stack the deck toward easy problems just to make my controller look good.
If anything, this failure rate is incredibly aggressive. A real production pipeline probably does not have a quarter of its steps permanently blocked. I will come back to this point later, because it is vital for interpreting the final results correctly.
The Baseline I Compare It Against
My point of comparison is a standard linear executor. It walks the tasks in a valid dependency order, attempts each one exactly once, and stops completely the first time it hits an obstacle it cannot immediately resolve. It allows no retries, no rerouting around blocked branches, and no partial credit for independent work that could still proceed.
I want to be explicit about one detail here. The linear executor does run tasks in a valid topological order rather than raw insertion order. This choice does not mean the baseline is being clever. It represents the absolute minimum requirement for this comparison to mean anything at all.
Without topological sorting, the linear executor would fail on task one of nearly any graph simply because its dependency happened to be listed later in the file. That would test list-ordering luck instead of linear execution. Past that baseline floor, the executor gets nothing extra. This is the honest, undefended default behavior of code that completely lacks a control loop.
The Bug That Nearly Invalidated the Whole Benchmark
I want to walk through this mistake in detail. Catching this specific bug is the exact difference between a benchmark you can trust and one that quietly lies to you.
My first implementation of the resource lookup returned a plain boolean. It returned True if a resource was available and False if it was not. The problem was that False was doing two completely different jobs. It meant not available yet, but check again next iteration for a resource that would resolve after a few polls. It also meant never going to exist, don’t bother checking again for a resource that was permanently missing. The controller had no way to tell those two scenarios apart from the return value alone.
This flaw had a major consequence. A task waiting on a slow but eventually available resource would sit in a needs resource state for a pass or two with no visible status change. My original termination logic would then trigger prematurely. That logic declared the whole graph deadlocked if nothing changed status in a full pass. As a result, it mislabeled a task that was still legitimately making progress as permanently stuck.
I caught this because my test suite included a case for exactly this scenario, and the test failed. To fix it, I changed the resource and decision lookups to return a three-state signal instead of a binary boolean: RESOLVED, PENDING, or MISSING.
Only a MISSING status proves a permanent block. A PENDING status means keep waiting. With this change, the loop’s iteration budget, rather than a heuristic guess, eventually decides when to give up.
I then wrote a regression test and a dedicated sanity check specifically to confirm that this class of bug cannot happen again. I ran every benchmark configuration at a normal iteration budget, ran it again at a budget forty times larger, and confirmed that the set of tasks marked permanently deadlocked remained identical both times. If a bigger budget ever changes the outcome, you are still guessing something rather than proving it. Across nine different configurations, the outcome never changed.
I include this because it’s the most honest thing I can show you about how this benchmark earned the right to be trusted. Shipping a boolean where you actually need three states is an easy, quiet mistake to make. It produces numbers that look completely reasonable right up until you check them against ground truth.
Sanity-Checking the Benchmark Before Trusting a Single Number
Most benchmark writeups I read, including some of my own earlier pieces, stop at the phrase here is the number. I wanted this specific benchmark to survive a skeptical read. Before I locked in a single number, I ran five specific checks against the benchmark’s own assumptions.
1. Do the injected failure modes match the intended design rate? I aggregated the data across 200 seeds and 4,800 task-mode assignments. Every single one of the six injected behaviors landed within half a percentage point of its intended frequency. The two permanent-block modes combined to exactly 25.2% of assignments, which perfectly validates my intended target of 25.0%.
2. When a task deadlocks, can I trace every downstream failure back to a real, injected blocker? I looked at a representative run where I directly injected 11 tasks as permanent blockers. I saw 9 additional tasks deadlock downstream of them. I successfully traced every single one of those 9 back through the dependency graph to one of the original 11. I found zero unexplained deadlocks.
3. Do retrieval, ask, and revise events happen in the expected proportions? I actually found a second, smaller issue here that is worth disclosing. My first version of this check expected exact equality between the number of injected slow-resource tasks and the number of successful retrievals I observed. The test failed. The explanation turned out to be completely correct and unremarkable. A handful of slow-resource, answerable-decision, and flaky tasks sat downstream of a different task that deadlocked first. The controller simply never reached them. I verified that cascade blocking fully explained every single shortfall, rather than the controller missing a task it should have completed. After that, the check passed cleanly.
4. Does giving the controller a larger iteration budget change which tasks it marks deadlocked? It does not. I tested nine different configurations at 50 iterations versus 2,000 iterations. This served as my direct regression test for the boolean versus tri-state bug I described earlier.
5. Are the results stable, or did I just get a lucky sample? I encourage you to weigh this check the most heavily, because most benchmarks skip it entirely. I tested across 300 random seeds.
Goal-directed controller completion %: mean = 47.7 stdev = 11.8 min = 11.6 max = 74.4
Linear baseline completion %: mean = 2.1 stdev = 2.7 min = 0.0 max = 14.0The variance is real and you should sit with it for a moment. The controller’s absolute worst seed completed 11.6% of its tasks. That number does not beat the linear baseline’s absolute best seed of 14.0%.
The Results, Led by the Metric That Actually Matters
I want to correct something about how I almost framed this benchmark. My first instinct was to lead with raw task completion rates. In a nine-configuration sample, the controller hit 46.7% completion against 8.3% for the linear baseline. Across the full 300-seed run, those numbers shifted to 47.7% against 2.1%.
Those are real numbers, but leading with them invites exactly the kind of inflated interpretation I am trying to avoid. It tempts you to say something like the controller is 20 times better, which is not what this mechanism actually demonstrates.
The metric that truly isolates the architectural claim is the number of independent branches fully completed:
Branches fully completed (avg across 9 seeded configs)
Goal-directed controller ################ 3.3 / 10.3
Linear baseline ## 0.4 / 10.3The controller was not better because it solved impossible tasks. It was better because it refused to let a single impossible task stop every other possible task. That is the entire contribution, stated as precisely as I can put it. I would much rather you remember that sentence than any single percentage.
Here is what that looks like iteration by iteration, using one representative configuration of 10 branches that run 4 tasks deep:
| Iter | Done | Blocked | Waiting | Retrieved | Asked | Revised |
|---|---|---|---|---|---|---|
| 1 | 6 | 12 | 25 | 0 | 3 | 2 |
| 2 | 12 | 15 | 16 | 2 | 1 | 1 |
| 3 | 19 | 17 | 7 | 0 | 0 | 1 |
| 4 | 20 | 17 | 6 | 2 | 0 | 0 |
| 5 | 22 | 20 | 1 | 1 | 0 | 0 |
| 6 | 23 | 20 | 0 | 0 | 0 | 0 |
Watch the Waiting column drain from 25 to 0 while Done climbs from 6 to 23. Notice how Retrieved and Asked spike in the exact iterations where the controller resolved a resource or a decision that had been blocking a branch.
A linear executor produces exactly one line total. Across the nine configurations I tested, it halted on the second or third task of the first branch in five of them, and on the first task of the second branch in the other four, never reaching a majority of the branches in any single run, regardless of whether they would have succeeded cleanly.
I want to be precise about that last point because it explains most of the gap in the numbers. The size of this gap is partly a property of my scenario design, which enforces one hard stop and zero further attempts. It is not a claim that the controller is broadly more capable.
What the controller demonstrably does is convert a catastrophic, all-or-nothing failure into a partial, isolated one. That represents a real, useful engineering property. It is absolutely not the same claim as calling the system smarter.
What This Benchmark Does Not Show
To avoid overselling these results, let me be clear. The controller does not complete 100% of the tasks, and it is not supposed to. Permanently missing resources and genuinely unanswerable decisions stay unresolved in both systems. No control-flow pattern, whether deterministic or model-driven, can retrieve a resource that does not exist or answer a question with no available answer. About half of the tasks in a typical run stay unresolved for exactly that reason.
This is also not a claim that deterministic control loops are sufficient for real agentic systems. Most production agent loops need a model or a human making judgment calls at some step. This is precisely why my architecture isolates that step as a single, swappable function call instead of baking a specific reasoning strategy into the control flow itself.
Finally, this is not a benchmark of loop engineering as a whole. That term covers scheduling, persistent memory, verifier-generator separation, and several other concerns that this small system does not touch at all. What I have demonstrated here is a single property, which I isolated on purpose so you could check it.
How to Tell If Your Own Pipeline Needs This Pattern
If you are building agent pipelines and wondering whether any of this applies to you, here is the honest checklist I use, based on what building this system taught me.
You probably have a linear-executor problem if you match these signs:
- A single failed step anywhere in your pipeline takes down every step after it, including steps that have zero dependencies on the one that failed.
- Your retry logic is a blanket try the whole pipeline again approach rather than a system that identifies which specific step needs another attempt.
- You cannot answer the question which independent parts of this job actually finished after a partial failure, because your logging only tells you where execution stopped.
- Adding a new independent branch of work to the pipeline means you have to modify the failure handling for every existing branch, instead of the new branch just plugging into the same control loop.
You can safely skip a goal-directed controller if your situation looks like this:
- Your pipeline is entirely sequential. If every single step genuinely depends on the one right before it, you have no independent branches. When there is nothing to isolate, failure isolation is useless.
- The engineering overhead is too high. Don’t build and maintain a complex state machine if rerunning a short pipeline from scratch is cheap. If you have a five-step job that finishes in under a second, stick to a linear executor with a simple top-level retry. It’s just more maintainable.
- Your actual bottleneck is judgment quality. This pattern won’t fix poor accuracy or bad decisions at a specific step. It only manages what happens after a step fails. That is a reasoning problem, not a control-flow problem. No amount of clever loop design will make your model smarter.
If you actually want to build this, three specific design choices will make or break the system. Get these wrong, and they will absolutely bite you in production.
Here is how they rank by potential damage:
- Never infer status from behavior. You have to distinguish “still working” from “permanently stuck” directly at the data level. A boolean return from a readiness check is an absolute trap. If a single
Falsemeans both a temporary delay and a permanent failure, your system can’t make smart choices. Use an explicit third state. - Don’t rely on termination heuristics. It’s easy to assume nothing changed in the last pass, so nothing will ever change. That’s wrong. It fails exactly when you need it to work: polling, backoffs, or waiting on external APIs. Your loop needs to track hard state, not timing guesses.
- Isolate the reasoning step. Wrap whatever decides the next move behind a single, swappable interface. The core control flow shouldn’t care if it’s talking to a Python rule, an LLM call, or a human callback. Keeping this boundary completely clean is the only reason I could benchmark the architecture in isolation.
Resources
- Addy Osmani, “Loop Engineering,” June 2026. The essay that named and structured the pattern.
- Let’s Data Science. Engineers Embrace Loop Engineering for AI Agents. Reporting on Boris Cherny’s CNBC interview (via Business Insider), Peter Steinberger’s X post, and related commentary.
- Geoffrey Huntley, “Ralph Wiggum as a ‘software engineer,’” July 2025. The original post describing the Ralph technique.
- Tosea.ai, “What Is Loop Engineering? A Complete Guide from Prompt to Harness Engineering,” June 16, 2026. Source of the prompt-to-context-to-harness-to-loop layer progression referenced above.
- DEV Community field guide synthesizing practitioner sources on agentic loops, including the Ralph technique and related work.
- Adnan Masood, “Loop Engineering: A Guide for Engineers and Practitioners,” Medium, June 24, 2026.
Disclosure
All code in this article was written by me and is original work, developed and tested on Python 3.12.10. Benchmark numbers are from actual runs on my local machine (Windows, CPU only) and are reproducible by cloning the repository and running benchmark.py and sanity_check.py, except where the article explicitly notes a number is a design target rather than an observed result (verified against actual rates in the sanity check). This project has zero external dependencies; all functionality runs on the Python standard library only, with no LLM calls anywhere in the benchmark itself. I have no financial relationship with any tool, library, or company mentioned in this article.
