The Long Prehistory of the Agent Loop (1966-2021)

Chapter 1 The Long Prehistory of the Agent Loop (1966-2021)

The machine in the Stanford Research Institute laboratory had been given a name that promised movement, but most of the time it stood still. Shakey was a steel frame on wheels, topped by a television camera and an optical range finder, wired by radio to a computer that filled much of the room. In 1969 researchers watched it confront a small world of flat wooden blocks and low platforms.

To move a block from one part of the room to another, Shakey did not simply roll forward. It stopped, took a picture, sent the image to the computer, and waited. The computer built a model of the room, searched through possible sequences of action, and transmitted a command. The robot then advanced a few inches and stopped again.

This halting rhythm was not a defect. It was the first working version of the loop that would eventually become the agent harness: sense, plan, act, repeat.

But the loop was incomplete. Shakey had no reliable way to check whether its action had produced the intended effect. A camera could misread a distance.

A plan could be logically sound but physically impossible, such as a command to roll a block through a gap narrower than the block itself. The robot could collide with a block, or freeze in a cycle of re-planning when its internal model disagreed with the room.

Researchers at SRI kept logs of each failure. Those records, kept as the project ran from 1966 into the early 1970s, are the first systematic documentation of an autonomous machine meeting reality and losing the argument.

Shakey had perception, planning, and action. It had no verify step. It had no ground truth.

This story opens not with a language model but with a loop. Charles Rosen, who directed the project, and Nils Nilsson, who shaped its planning methods, had decomposed autonomy into modules that could be improved separately. That division gave the field its first clean architecture. In 1971 Richard Fikes and Nilsson formalized the planning component as STRIPS, a language that described actions as operators with preconditions and effects.

To push a block, the robot had to be next to it and the path clear; after pushing, the block occupied a new location.

STRIPS did not solve Shakey’s movement problem. It made the problem precise enough to study. It turned the plan step into a discipline with its own algorithms, journals, and benchmark problems.

What STRIPS could not do was tell the planner whether the world had obeyed. A sequence of operators could be flawless on paper and fail on the first motion because a block had shifted or a doorway had narrowed. Without a signal that said worked or failed, the loop was blind. The agent could sense and plan and act, but it could not learn.

This was not a failure of intelligence. It was a failure of the harness: the scaffolding that connects a reasoning core to reality and tells it whether the action landed. The classical planning tradition that grew from STRIPS spent decades refining this structure. Planners learned to handle partial observability, probabilistic outcomes, and time.

But they remained confined to blocks worlds, logistics puzzles, and other domains small enough to model by hand.

The reason was not that the planners lacked cleverness. They lacked ground truth. In a robot that pushes blocks, the only feedback is the same camera that failed in the first place. In a planner that arranges deliveries, the result arrives days later, through forms and invoices. The loop never closes fast enough to guide the next attempt.

The ground-truth law would later state the condition in reverse: agents land first where feedback is verifiable, and drift where it is not. Shakey was the first proof.

While Shakey labored in California, Joseph Weizenbaum at MIT built a different kind of machine, one that had no body and could not act. ELIZA, completed in the mid-1960s, held a typed conversation by matching patterns in a user’s sentences and filling in scripted responses. A user who wrote “I am unhappy” might receive “Why do you say you are unhappy?”

The program had no model of the speaker, no goal, and no way to verify whether its response helped.

Yet people confided in it. Weizenbaum watched as his secretary asked him to leave the room so she could speak privately with the machine.

ELIZA lacked every part of the Shakey loop except the production of text. It had no ground truth, and because it had no ground truth, there was no measure of whether it was working. That did not stop users from trusting it.

The early lesson was double-edged: an autonomous loop without verification drifts, while an interface without verification can still collect trust it has not earned. The metering of trust—the later discipline of selling autonomy in units of trust rather than intelligence—had not yet been invented. Its absence was already visible. At Bell Labs in Murray Hill, New Jersey, the same years produced a different kind of loop, built not around a robot but around a terminal.

Ken Thompson and Dennis Ritchie were constructing Unix, and Douglas McIlroy had pressed for a new way to assemble work.

Each program should do one thing well. Each should consume a stream of text and produce a stream of text. The pipe character joined them. A user could type a command that listed files, passed the list to a search program, and passed the matches to a counter. The full task existed nowhere as a single program; it emerged from the composition of small, predictable tools.

This was tool composition at the level of an operating system. It required no intelligence from the tools. It required a contract. Each tool had clear inputs and outputs, which made it replaceable with any other tool that respected the same contract. The insight would become the unacknowledged ancestor of tool-calling APIs. When a later agent needed to execute a search or read a file, it would not need to know how the search worked; it would only need to know the contract—the input format, the output format, and the side effects.

That contract was the harness. Unix did not invent the idea of modularity, but it gave the idea a syntax and a culture. By the 1980s, the pipe had become the default mental model for a generation of programmers. When they faced a complex task, they reached for a chain of small programs rather than a single large one.

The limitation was always the same: a human had to choose the chain. The operating system offered no mechanism for a program to decide, mid-execution, that it needed a different tool and to invoke it. The loop was there—execute, capture, feed forward—but the decision step was missing. The Unix lineage had built the plumbing; it lacked the brain.

The third lineage came from a community that rarely thought about artificial intelligence at all. In the late 1980s, Kent Beck was working on a Smalltalk project and grew tired of manually verifying that his code still worked after each change. He began writing small automated checks that exercised a piece of code and reported a simple result: pass or fail.

He called them unit tests. By 1994, he and Erich Gamma had packaged the practice into a framework called SUnit, which introduced the now-familiar red bar and green bar. A green bar meant all tests passed; a red bar meant something was broken. It was a binary signal of ground truth. The xUnit family that followed—JUnit for Java in 1997, then NUnit, PyUnit, and dozens of others—spread this practice across the software industry.

Continuous integration servers began running test suites automatically every time a developer committed code. By the early 2000s, millions of programmers were working inside a feedback loop that was fast, unambiguous, and ruthless. You wrote code, you ran tests, you saw red or green. If it was red, you fixed the code and ran again.

This was not merely a productivity tool; it was an industrial infrastructure for verifying correctness. It turned programming into a domain where an agent could know, with high confidence, whether its action had succeeded. The testing lineage had solved the problem that stranded Shakey.

It provided the missing “verify” step in the sense-plan-act-verify loop. But it solved it only for the narrow world of software, where the environment is made of deterministic rules and the feedback signal is perfectly clean. A unit test either passes or it fails; there is no ambiguity about whether a block was shoved far enough. This is why, decades later, code generation would become the killer domain for AI agents. The ground truth was already there, waiting.

These three lines of development—planning, tool composition, and verifiable feedback—grew in parallel through the closing decades of the twentieth century, each advancing its own piece of the puzzle without any awareness of the others. The classical AI community continued to refine planners like STRIPS into systems that could handle partial observability and probabilistic outcomes, but they remained confined to toy domains. The Unix philosophy spread through the open-source movement and became the default mode of building server software, but it never acquired an intelligent orchestrator.

The testing culture conquered professional programming and made automated verification a baseline expectation, but it was always driven by human-written tests for human-written code.

Occasionally, someone would try to connect two of these threads. In the 1980s, the expert systems boom produced inference engines that could chain rules together to reach conclusions—a kind of planning loop married to a knowledge base. Systems like MYCIN, which diagnosed bacterial infections, and XCON, which configured VAX computer systems for Digital Equipment Corporation, demonstrated that a machine could reason through a complex domain if enough rules were encoded by hand. But these systems were brittle and expensive to maintain. Their inference engines were proprietary, locked inside each vendor’s shell. There was no standard interface for plugging in new knowledge or new tools. This was an early lesson in protocol politics: whoever controlled the inference engine controlled the ecosystem, and no one wanted to cede that control.

The expert systems market collapsed in the late 1980s not because the idea of automated reasoning was wrong, but because the harness was too rigid and too closed. The scaffolding paradox was already at work, though no one called it that yet. The inference engines built for expert systems were eventually absorbed into general-purpose programming languages and business rules engines. What had been a specialized harness for medical diagnosis became a feature of enterprise software platforms. The harness builders of the 1980s—companies like Teknowledge and Intellicorp—found themselves competing against tools that made their core innovation a commodity. They had climbed one level of the stack only to have the ground rise up beneath them.

Trust was another unresolved tension. Even when expert systems worked, users were reluctant to hand over decisions to a machine that could not explain itself in terms a human found persuasive. MYCIN could recommend an antibiotic regimen with accuracy rivaling a specialist, but doctors wanted to know why.

The system’s chain of rules was technically transparent—you could inspect every inference step—but it was not legible in the way a colleague’s reasoning would be.

This was an early instance of what would become a central problem for agent harnesses: autonomy is sold in units of trust, not intelligence. Half the engineering effort goes not into making the model smarter but into making humans willing to let go. The metering of trust would become explicit only decades later, when permission prompts and sandboxes and human-in-the-loop checkpoints became standard features of agent frameworks. But its roots are here, in the hesitation of a doctor staring at a MYCIN printout and deciding to override it.

By the turn of the millennium, the pieces of the agent harness were scattered across the computing landscape like unconnected circuit elements. The planning tradition had given us algorithms for decomposing goals into subgoals and ordering actions to achieve them. The Unix tradition had given us a model of composable tools with clean interfaces.

The testing tradition had given us an infrastructure of ground truth that could close the loop with a binary signal. And the expert systems episode had provided a cautionary tale about proprietary protocols and the slow work of building trust.

What none of these traditions had was a component that could sit at the center of the loop and make sense of the world in flexible, general terms. The planners needed a symbolic model hand-coded by engineers. The Unix pipes needed a human to decide which programs to chain. The test suites needed a human to write the tests and interpret the failures. Each system had a gap where general intelligence was supposed to go.

Then, in the second decade of the twenty-first century, that gap began to close from an unexpected direction. Neural networks, which had been dismissed by the classical AI community as statistical tricks with no capacity for reasoning, started producing results that were impossible to ignore.

In 2012, a deep convolutional network called AlexNet crushed the competition in an image recognition contest, and suddenly the old symbolic approach to perception looked obsolete. Language models followed a similar trajectory: first word embeddings that captured semantic relationships, then recurrent networks that could generate plausible sentences, then transformer architectures that scaled with astonishing predictability. By 2020, OpenAI had released GPT-3, a language model with 175 billion parameters that could produce coherent paragraphs of text on almost any subject.

It was not an agent. It had no loop, no tools, no ground truth. It was an oracle: you gave it a prompt, and it returned a completion.

But for the first time, there existed a component that could potentially fill the hole at the center of the old harness architecture. A model that could read instructions in natural language, reason about them in ways that mimicked planning, and generate commands that looked like tool invocations. The arrival of this capability did not immediately trigger a revolution in agent design.

. The model would generate a function call with arguments, the system would execute it, and the result would be fed back into the model. This was a direct descendant of the Unix pipe: small, single-purpose tools with clear inputs/outputs. The model became the orchestrator that Unix had lacked. Early experiments with “program-aided language models” (PAL) or “toolformer” (2023) but we can mention the conceptual lineage. The model’s inability to reliably format calls was a major hurdle; the trust problem again.

Paragraph 3: The planning lineage resurfacing. The idea of decomposing a goal into subgoals, searching through possible actions, had been the hallmark of STRIPS. With language models, the planning was implicit in the prompt, but it was unreliable. Early attempts to use the model as a planner, generating a sequence of actions, then executing them.

The agent loop of ReAct (reasoning and acting) was a synthesis of planning and tool use. The “ReAct” paper was published in 2022, but the ideas were in the air in 2021. Labs like Google Brain and DeepMind were exploring “language model programs” and “policy” with tools.

We can mention the “WebGPT” paper (2021) that used a model to browse the web, which is a loop: search, click, read, repeat. That’s a direct agent loop. WebGPT is from 2021, so it’s within the prehistory.

We can describe it: a model that could issue search queries, scroll through results, and gather information to answer a question, all in a loop of perception and action. That was a proto-agent.

Paragraph 4: The trust problem intensifies. As these loops were built, the unreliability of the model became the central challenge. The model would sometimes generate a tool call that was malformed, or it would get stuck in a loop, or it would produce answers that sounded confident but were wrong. The early builders realized that the harness needed not just the loop but also guardrails: sandboxes, verification steps, and explicit trust signals. The red/green test from the testing lineage became a mental model for how to ground the agent’s output. People started writing tests for the agent’s behavior, not just the code it generated. The idea of “evals” for agents emerged. This directly connects to the earlier history of expert systems and the MYCIN trust issue: the same problem of transparency and trust, now with a black-box model.

Paragraph 5: The institutional landscape. By the end of 2021, a few startups and research groups were openly working on agent frameworks. OpenAI’s own API had introduced function calling later, but the community was building on top. The “LangChain” project started in October 2022, but the precursor ideas were swirling. We can mention the “AutoGPT” explosion in 2023 as a later chapter. For now, we can say that the momentum was building, and the three lineages were finally converging in a haphazard, experimental way. The chapter ends with the race about to begin, so we can set the stage: the old machinery of planning, pipes, and tests was being dusted off and retrofitted to the new kind of intelligence. The question was not whether a loop could be built, but whether it could be made reliable enough to trust with real-world tasks.

We’ll write these paragraphs in a flowing, book-prose rhythm, with concrete details where possible (WebGPT, specific early tools, the echo of Shakey’s failures). Avoid bullet points, no meta commentary. We’ll output only the new paragraphs, not the original draft.

Word count: I’ll aim for about 1900 words. Let’s draft.

New paragraphs:

The first attempts to close the loop with GPT-3 were crude, often no more than a script that fed the model’s own output back into the prompt and asked for a revision. In the months after the API opened in mid-2020, a scattered community of prompt engineers discovered that the model could be coaxed into correcting its own mistakes if you told it to “check your work” or “list the steps and then answer.” The results were erratic. Sometimes the model would spot a contradiction and fix it; more often it would double down on a hallucination with fresh confidence. But the pattern was unmistakable: a loop was forming, even if it was only a loop of text.

The model would generate, a human would inspect, and the human would feed the output back with a new instruction. This was not yet autonomy; it was a conversation. But it taught the early experimenters a crucial lesson: the model needed scaffolding, not just a better prompt. The sense-plan-act-verify cycle that had been dormant since Shakey was stirring again.

In the code-generation domain, the loop closed with unusual speed. When OpenAI released Codex, a version of GPT-3 fine-tuned on publicly available code, in August 2021, developers immediately began building tools that would take a natural-language description, generate a function, run it against a test suite, and feed any failure messages back to the model for a fix. The red/green bar that Kent Beck had pioneered in the 1990s became the feedback signal. A model that could write code that passed tests was a model that could be trusted to produce workable output, at least in this narrow domain. This was the first practical demonstration that the old harness machinery could make a language model useful. The testing lineage, which had solved the problem of verifiable feedback for human programmers, proved to be just as valuable for machine-generated code. It was not a coincidence that code generation became the killer application for early agents: the ground truth was already purchased, installed, and running on millions of machines.

Meanwhile, the tool-composition lineage was resurfacing in a different form. Programmers who had grown up with the Unix pipe instinctively reached for the same pattern when they connected GPT-3 to external systems. They would describe a set of tools—a search engine, a calculator, a database query—as functions with clear inputs and outputs. The model would then be prompted to decide which tool to call and with what arguments. The output of the tool would be injected back into the prompt, and the model would continue. This was the unacknowledged ancestor of tool-calling APIs: the model was the brain, and the tools were the pipes.

The early implementations were fragile. The model would often generate a function call that was syntactically incorrect, or it would hallucinate a tool that did not exist. But the idea was powerful enough that by late 2021, several research groups had published proof-of-concept systems that could answer questions by retrieving documents from a search engine, or perform arithmetic by calling a calculator.

The most advanced of these was WebGPT, a project from OpenAI that gave the model the ability to issue search queries, scroll through results, click links, and synthesize findings into a summary. The system operated in a loop: it would read a query, decide whether to search or scroll, execute the action, and then feed the new page content back into the model. This was a direct descendant of Shakey’s sense-plan-act rhythm, but instead of navigating a room of blocks, it was navigating the open web. The loop was the same; the domain was a thousand times larger.

WebGPT’s published results in December 2021 showed that the model, when augmented with a search tool and a verifier, could produce answers that were more factual and more useful than the model alone.

But the paper also revealed the persistent trust problem. The model would sometimes click on a link that looked promising but led to a dead page, and it would then invent a plausible-sounding answer based on the link text alone. It could get stuck in a loop, scrolling endlessly through search results without synthesizing a response. The researchers had to introduce a “stop” action and a “no action” option to keep the agent from spinning out.

These were not failures of intelligence; they were failures of the harness. The loop was working, but the verification step was still too weak. The model needed a clearer signal of whether its action had succeeded, and the web—unlike a unit test suite—did not provide one. The dream of a un

The planning lineage, which had been absent from these early tool-use experiments, was also beginning to stir.

At Google Brain and DeepMind, researchers were exploring whether a language model could be prompted to generate a sequence of actions—a plan—before executing any of them. The model would be asked to think step by step, listing the subgoals and the tools required, then carry out the steps in order. This was a direct echo of STRIPS, but with a critical difference: the plan was not a formal operator sequence but a natural-language script that the model itself had to interpret.

The results were mixed. Sometimes the model would produce a coherent plan and follow it faithfully; other times it would skip steps, invent tools, or lose track of the goal mid-execution. The old planning algorithms had been designed for a world of symbolic certainties; the new planners were operating in a world of probabilistic text. The mismatch was a source of both frustration and fascination.

It suggested that the harness would need not just a loop but a way to enforce the plan’s structure, to verify each step, and to recover when the model wandered.

This recovery problem became the central engineering challenge of the period. When a Shakey-era planner failed, it could re-plan from a known state. When a language model agent failed, it often had no reliable state at all. Its memory was the prompt, a growing accumulation of text that could become contradictory or overwhelming. Builders experimented with summarizing past actions, truncating the context, and injecting explicit state variables. Each solution was a patch on a deeper wound: the model had no native concept of a world that persisted independently of its text. The trust problem that had haunted MYCIN returned in a more virulent form. A doctor could inspect MYCIN’s rule chain and decide whether to trust it; a developer staring at a GPT-3 agent’s trace could see only a sequence of plausible-sounding sentences. The black-box nature of the model meant that trust had to be built externally, through sandboxes that limited what the agent could do, verifiers that checked its outputs against ground truth, and human-in-the-loop checkpoints that required approval before high-stakes actions. The metering of trust, which had been an implicit undercurrent in the expert systems era, was becoming an explicit design requirement.

The metering of trust became a design obsession. Builders learned that they could not simply ask the model to perform an action and hope for the best; they had to insert checkpoints where a human could review the plan, approve the tool call, or inspect the output. This was a direct descendant of the hesitation that had made Weizenbaum’s secretary trust ELIZA and made doctors override MYCIN. The interface had to signal that the machine was not acting autonomously but was offering a proposal for human ratification. Even in fully automated loops, the trace of those checkpoints remained as log messages and audit trails.

The loop was not just sense-plan-act-verify; it was sense-plan-act-verify-trust. And trust, unlike verification, could not be automated. It had to be earned over time, through a history of reliable behavior. The prehistory of the agent loop thus ended with a paradox: the more capable the model became, the more elaborate the harness needed to be to keep it trustworthy. The old machinery of planning, pipes, and tests was being retrofitted not to amplify intelligence but to contain it.

By the end of 2021, the outlines of a new kind of software were visible in the scattered experiments of a few dozen research labs and open-source projects. The community that formed around these experiments was small but intensely active.

On forums and in Discord servers, developers shared prompts that made the model more reliable, scripts that chained multiple calls, and horror stories of agents that had sent embarrassing emails or deleted files. The mood was a mixture of exhilaration and caution. Everyone could see the potential; everyone could see the fragility.

The scaffolding paradox that had plagued earlier harness builders—that the harness itself becomes the product, then gets commoditized—was already in the air. Startups that built tool-calling wrappers around GPT-3 found themselves in a race against the API providers, who could add the same functionality natively with a single update. The lesson of Teknowledge and Intellicorp, who had watched their expert system shells become features of larger platforms, was being repeated in real time.

For nearly two years after GPT-3’s release, most developers treated it as a curiosity—a text generator that could write poems and answer trivia but could not be trusted to do real work. The model hallucinated facts, lost track of context in long conversations, and had no way to interact with external systems. It was a brain in a vat.

But a small number of researchers and engineers began to see something else. They saw that if you wrapped the model in the right scaffolding—a loop that fed its outputs back as inputs, tools it could call to affect the world, tests that could verify its results—you might transform an oracle into a worker. The old machinery of sense-plan-act-verify had been waiting for decades. Now it had a candidate intelligence to drive it. The pressure point was not that language models had become smart enough to replace the harness. It was that they had become just smart enough to make the harness necessary. A model that could talk but not act was a fascinating toy.

A model that could talk and also push blocks—or send emails, or query databases, or run code—was something else entirely. The question hanging over the field in 2021 was whether anyone could build a harness that channeled this raw linguistic capability into reliable action without breaking trust or drowning in complexity. The prehistory was over. The loop had found its missing piece. And the race to build the harness was about to begin.