The Prompt Engineers

Chapter 2 The Prompt Engineers

The question hanging over the field in 2021 was whether anyone could build a harness that channeled the raw linguistic capability of large language models into reliable action without breaking trust or drowning in complexity. The answer arrived piecemeal, not in a single lab or a white paper, but in a thousand text editors, late at night, when developers who had seen what GPT-3 could do began to ask what it could become.

The first answer was a commit. In a small apartment in San Francisco, Harrison Chase stared at his terminal. He had been experimenting with GPT-3 for weeks, feeding it prompts and watching it spin out plausible but unreliable text. The model was a brilliant oracle, but oracles are not workers. Tonight, he was trying something different. He had written a short Python script that took the output of one API call, parsed it, and fed it into a second call, along with a new instruction. The model would no longer answer a single question; it would follow a chain of reasoning that he had programmed into the loop.

The script was crude—a few dozen lines of code—but it worked. The model seemed to think more clearly when its own output was piped back into its input.

Chase committed the code to a new repository he called LangChain. The date was October 17, 2022. The commit message was terse: “initial commit, simple sequential chain.” There was no grand vision, no product roadmap. The repository was a side project, a way to share a trick with other developers. But that trick contained the seed of an entire industry.

The loop—generation, observation, action—had been born not from a research lab but from the practical necessity of a frustrated engineer who needed a language model to do more than talk. A few days earlier, on October 6, a preprint had appeared on arXiv that gave the trick a name. “ReAct: Synergizing Reasoning and Acting in Language Models” was authored by Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. The paper proposed a simple but powerful pattern: interleave reasoning traces with actions.

The model would generate a thought, then an action (like using a calculator or searching Wikipedia), then observe the result, and then continue reasoning. The authors demonstrated that this interleaving dramatically improved the model’s performance on knowledge-intensive tasks, because the external feedback could correct the model’s hallucinations. The ReAct pattern was not invented by the paper; it was observed and formalized. By the time it appeared, dozens of developers had already stumbled onto similar ideas, but the paper gave them a shared vocabulary.

It was the moment the harness layer began to self-consciously exist. The first harness layer was not designed. It was improvised.

And it was improvised by people who treated the language model as a raw component, not a finished product. To understand why this improvisation was necessary, we have to go back a few months earlier, to the first great prompt-engineering discovery of 2022. In January, a team of researchers at Google—Jason Wei, Xuezhi Wang, and others—published a paper titled “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models.”

The paper showed that simply adding the phrase “Let’s think step by step” to a prompt could dramatically improve the model’s ability to solve complex reasoning problems. The model did not need to be retrained; it just needed to be prompted to reason. The discovery swept through the developer community. It was a harness in the smallest possible sense: a single sentence that altered the model’s behavior. But it revealed a deeper truth: the model’s raw intelligence was not enough; it needed a structure to guide it. The prompt was the first harness, and the “Let’s think step by step” trick was its earliest and most famous example.

Riley Goodside, a data scientist in San Francisco, became one of the most visible practitioners of this new craft. Goodside had been experimenting with GPT-3 since its launch, and he quickly realized that the model’s behavior could be shaped by careful prompt design. He began posting his findings on Twitter, sharing elaborate prompts that coaxed the model into performing tasks it had not been explicitly trained for.

He called himself a “prompt engineer,” and the term stuck. In early 2022, Goodside was hired by Scale AI as the world’s first staff prompt engineer, a sign that the craft had commercial value. His work was not just about clever phrasing; it was about building a harness around the model’s raw ability, turning it from a general-purpose oracle into a specialized tool. Goodside’s prompts were often long, multi-paragraph constructions that included examples, constraints, and a clear chain of reasoning. They were not programs in the traditional sense, but they were a form of code—a code that the model itself executed.

The community that formed around these techniques was scattered across GitHub, Twitter, and Discord. They shared prompt templates, compared notes, and built small libraries that chained prompts together.

The model was still an oracle, but the harness was beginning to take shape. Developers discovered that they could pipe the model’s output to a calculator API, and then feed the result back into the prompt. This was a quantum leap.

A language model that could not reliably add three-digit numbers could now invoke a calculator, read the correct answer, and incorporate it into its response.

The model had become a worker, not just a thinker. It could observe the world, act on it, and observe again.

The loop was the primitive of all future agent architectures. The loop was simple, but its implications were profound. A model that could use a calculator could also use a search engine, a database, a code interpreter. Each tool required a different interface, a different format for the model’s action and observation. The prompt engineers began to invent informal protocols for these interactions. A prompt might include a section like “If you need to search, output ‘SEARCH: <query>’ and I will return the result.” The model would then generate the search query, the developer’s script would execute it, and the result would be fed back in. This was a harness in the truest sense: a set of protocols and tools that surrounded the model and gave it new capabilities.

The protocols were ad hoc, and they varied from project to project, but they were the first draft of the agent layer.

The ReAct paper gave this practice a theoretical foundation. It showed that the interleaving of reasoning and action was not just a useful hack; it was a general principle that could be applied to many tasks. The paper’s experiments on HotpotQA and Fever demonstrated that the ReAct pattern outperformed both pure chain-of-thought and pure action-only baselines. The model could reason about what it needed to know, take an action to find out, and then integrate the new information. The paper was published just as the developer community was hungry for such a framework, and it was widely shared. GitHub repositories implementing the ReAct pattern began to appear, including one by a researcher named Yao Fu, and another by Chase himself.

The harness layer was no longer just a collection of tricks; it was becoming a discipline. The release of ChatGPT on November 30, 2022, supercharged the process. ChatGPT was not a new model; it was GPT-3.5 with a chat interface.

But the interface was a harness in itself, a harness that constrained the model to a conversational format and gave it a memory. The public was captivated, and developers rushed to build on top of the API. The number of GitHub repositories tagged “prompt-engineering” exploded. The prompt engineers were no longer a niche community; they were the vanguard of a new industry.

But the very success of the harness contained the seeds of its own absorption. The model vendors—OpenAI, Google, Anthropic—were watching. They saw the community’s inventions and began to incorporate them directly into the models. Chain-of-thought reasoning, which had been a prompt-engineering trick, was eventually baked into the model’s training data. Tool use, which had been an external harness, was integrated into the model’s API. The Scaffolding Paradox was at work: every harness invention would eventually be swallowed by the model itself, forcing the harness builders to climb one level higher or be rendered obsolete. The prompt engineers of 2022 were the first to feel this pressure.

The craft they had invented was already being studied by the very companies whose models they were wrapping. The threat was not immediate, but it was visible to those who looked closely.

In a blog post from early 2023, a developer named Shawn Wang warned that “prompt engineering is a temporary job.” The models would get better, and the prompts would become simpler. The elaborate chains that Goodside and others had built were at risk of being cleaned up by the next model update. The harness layer was a moving target. What was necessary today might be absorbed tomorrow.

Despite this looming shadow, the energy of the community was irrepressible. The months after ChatGPT’s release saw a frenzy of experimentation. Developers built prompt chains that could write essays, debug code, and even simulate entire conversations between multiple agents. The “BabyAGI” project, created by a developer named Yohei Nakajima, used a loop of prompts to create an autonomous task manager that could break down a goal into subtasks and execute them.

The project was a simple Python script, just a few hundred lines, but it became a sensation because it demonstrated the power of the loop. The model was not just a tool; it was an agent, capable of planning and acting on its own. The loop had become a harness that could sustain a chain of reasoning far beyond a single prompt.

The economic implications were not lost on investors. Venture capital began to flow into startups that were building on top of these harnesses. The promise was that the harness layer could be a new platform, a new operating-system layer for the AI era. But the startups were building on a foundation that was not their own. The model vendors held the key to the model, and they could change the locks at any time. The pressure was building: the nascent harness layer, now attracting venture capital, had to outrun the model vendors’ absorption.

The months after ChatGPT’s release were not just a frenzy of experimentation; they were a period of rapid codification. What had been scattered tricks in GitHub gists and Discord channels began to coalesce into shared libraries and reusable patterns.

The LangChain repository, which Harrison Chase had started as a side project, became a focal point. By early 2023, it had attracted dozens of contributors who added abstractions for memory, chains of prompts, and agents that could decide which tool to invoke next. The library did not invent new capabilities; it standardized the protocols that the community had been inventing in isolation. A developer in Berlin could now use the same SimpleSequentialChain class that someone in Tokyo had used the week before.

This standardization was itself a form of harness, a meta-layer that made the ad-hoc loops reproducible and shareable. It lowered the barrier to entry, and the number of projects built on LangChain exploded. The library became a kind of operating system for prompt engineers, but its very existence underscored the paradox: it was a scaffold that the model vendors might one day render unnecessary by building its functionality directly into the API.

The loop that powered these chains was deceptively simple, but its implications for error correction were profound. Early experimenters noticed that when a model was allowed to observe the result of its own action—especially when that result came from a reliable external source—it could sometimes detect and correct its own mistakes. A model asked to compute a complex financial projection might first output a flawed reasoning step, then, upon seeing the calculator’s result, revise its chain of thought. This self-correction was not reliable, but it was a glimpse of a new kind of system: one where the harness compensated for the model’s weaknesses. The ReAct paper had formalized this interleaving, but the community pushed it further. They built loops where the model could generate multiple possible actions, evaluate their outcomes, and backtrack. These were not yet autonomous agents, but they were more than simple prompt chains. They were early experiments in agency, and they were built entirely in the harness layer, with the model itself unchanged.

Yohei Nakajima’s BabyAGI project, released in early 2023, captured this ambition in a few hundred lines of Python. The script maintained a task list in a simple text file, used GPT-3.5 to generate new tasks based on a high-level objective, and then executed them in a loop. The model would create a plan, execute the first step, observe the result, and then reprioritize the remaining tasks. BabyAGI was not a product; it was a demo, a proof of concept that the loop could sustain goal-directed behavior over dozens of iterations. The project’s GitHub repository went viral, and within weeks, developers had forked it to add memory, tool integrations, and even a web interface. The excitement was palpable, but so was the fragility. The system could easily spiral into nonsense, generating tasks that were irrelevant or contradictory. The loop was a powerful engine, but it lacked the guardrails that a production system would require. The prompt engineers were learning that building a harness was not just about enabling capabilities; it was about containing them.

This tension attracted the attention of venture capitalists, who saw in the harness layer the potential for a new platform. Startups like Fixie, Dust, and others emerged to build enterprise-grade frameworks for chaining prompts and tools. Their pitch was that the raw model was not enough; companies needed a reliable way to integrate it into their workflows, with logging, access controls, and error handling. The harness, they argued, would be the middleware of the AI era. But this positioning put them on a collision course with the model vendors. OpenAI, Google, and Anthropic were not just selling raw models; they were building their own platforms, with plugins, function calling, and fine-tuning APIs that encroached on the territory the startups were claiming. The pressure was no longer theoretical. It was a business reality. The prompt engineers had built a craft, and now that craft was being pulled in two directions: upward into the model itself, and outward into a standalone industry. The choice between reckless autonomy and engineered frameworks was not just a technical one; it was an existential question for the companies betting on the harness.

Yet the community’s improvisational spirit persisted. Even as the frameworks matured, individual developers continued to push the boundaries of what a simple loop could do. On Twitter and in Discord servers, they shared prompts that made the model simulate entire software development teams, with one agent acting as product manager and another as engineer. These experiments were often brittle and unreliable, but they demonstrated a crucial point: the harness did not need to be perfect to be useful. A loop that worked 80% of the time could still automate tasks that had previously required human effort. The prompt engineers were not building bridges; they were building ladders, and they were willing to climb them even if a few rungs were missing. This pragmatism was both a strength and a vulnerability, because it meant that the harness layer was always provisional, always one model update away from obsolescence. The very improvisation that had given birth to the layer now made it hard to defend. The model vendors could study the community’s best tricks, implement them natively, and offer them with a reliability that the harness could not match. The prompt engineers of 2022 had shown what was possible; the question was whether they could build something that would last.

The fork in the road was visible: one path led to reckless autonomy, where the harness was pushed to its limits without regard for safety or reliability. The other path led to engineered frameworks, where the harness was carefully designed to be robust and trustworthy. The choice would define the next phase of the industry. The prompt engineers of 2022 had built the first harness with nothing but their own ingenuity and a willingness to experiment. They had turned a machine that could talk into a machine that could work. But the ground was shifting beneath their feet. The model vendors were already moving to absorb their inventions. The question was no longer whether a harness could be built, but whether it could survive. The first harness layer was not designed. It was improvised. And its improvisers were about to learn that improvisation is not enough to hold a platform.

The next chapter of the harness story would be written in the spring of 2023, when a new generation of developers would push the loop to its breaking point, and the industry would be forced to choose between the wild promise of autonomous agents and the hard engineering of frameworks. The prompt engineers had lit the fuse. The explosion was coming.