The Protocol Turn (November 2023–March 2024)

Chapter 5 The Protocol Turn (November 2023–March 2024)

The scaffold was built. Now it needed to bear weight, and the cracks were showing.

Years earlier, the pressure found its first concrete release not in a press release or a conference keynote, but in a series of silent, consequential updates to two lines of code.

In the late autumn of 2023, the engineering teams at OpenAI and Anthropic each shipped a version of the same fundamental idea. OpenAI’s iteration was called “function calling,” Anthropic’s “native tool use.” The technical specifics differed slightly in syntax, but the semantic shift was identical: the model’s API would now formally accept a list of tools—their names, descriptions, and parameter schemas—and the model itself would decide when and how to call them, returning a structured JSON object instead of a line of prose. A developer no longer needed to write parsing logic to extract “search(“San Francisco weather”)” from a sentence like “Could you look up the weather in San Francisco?” The model would output the structured call directly.

It was a quiet, almost surgical absorption of a layer that had, just months prior, supported a thriving ecosystem of third-party wrappers, frameworks, and startups. The ground, meticulously laid by the framework rush, liquefied underfoot.

This was the scaffolding paradox in its purest, most accelerated form. Every harness invention—the tool-calling logic, the structured output, the very concept of an agentic loop—was a feature the model vendors were already training into the next checkpoint. The summer’s sprawling abstractions had not been wrong; they had been prophetic. They had correctly identified the necessary components of a working harness: a loop, a set of tools, a plan.

But by proving these components were not just possible but essential, they had handed the model vendors a ready-made blueprint. The frameworks had built the scaffolding, and now the model was learning to climb it.

For the companies that had staked their existence on selling that scaffolding, the only viable response was to abandon the layer being absorbed and climb one level higher. The protocol turn was that climb.

It was the industry-wide pivot from building on top of models to building between agents and the world, defining the languages and rules that would govern their interaction even as the models beneath them continued to evolve.

The rapid decline of LangChain’s centrality became the most visible symptom of this pressure. Throughout 2023, LangChain had ridden the framework rush to staggering prominence. Its repository had amassed tens of thousands of stars, its abstractions—“chains,” “agents,” “tools”—had become the lingua franca of early agentic experimentation. Its valuation narrative, buttressed by venture funding, was that of “the agent OS,” a foundational layer upon which the future of automated work would be built.

But the function-calling API updates of late 2023 acted like a solvent on this narrative. Developers, always pragmatic, began to ask a simple question: if the model could now handle tool selection and structured output natively, what exactly was LangChain’s core abstraction for? The answer, for a growing cohort, was “less than before.”

This shift in focus was immediately evident in the startup pitches of late 2023 and early 2024. Where once founders sold “the best way to call GPT-4,” they now pitched “the only platform that can manage a thousand-agent workflow with audit trails.”

Companies like Smyth Systems and the open-source project CrewAI emerged not as alternative frameworks, but as something conceptually different: schedulers and supervisors for AI labor. Their core abstractions were not “chains” but “workflows,” “teams,” and “tasks.” They offered persistent memory that outlived a single API call, explicit state machines to manage multi-step processes, and built-in guardrails for quality control. Crucially, they were model-agnostic by necessity, designed to treat OpenAI, Anthropic, or a local Llama instance as interchangeable compute nodes.

This was the scaffolding paradox in action: having been cut out of the tool-calling layer, the harness builders ascended to the orchestration layer, where the problem was not making the model smart, but making a system of models reliable, scalable, and trustworthy.

The architectural choices of these new platforms revealed the nascent priorities of a production-grade harness.

Smyth’s early white papers, for instance, emphasized “deterministic orchestration,” a direct response to the inherent nondeterminism of the underlying models. A workflow in Smyth was defined as a directed graph where each node’s execution—calling a model, a tool, or a human review—was logged, its inputs and outputs hashed, and its failure modes cataloged for automatic retry with alternative parameters. This treated the AI not as an oracle but as a fallible, if powerful, subroutine.

Similarly, CrewAI’s metaphor of “agents” as specialized workers in a “crew” forced developers to think in terms of role-based delegation and handoffs, implicitly designing processes where a single monolithic AI call was insufficient. Both approaches formalized something the earlier framework rush had only gestured at: the metering of trust was moving from an afterthought to a primary product feature. These platforms sold control.

This ascent to orchestration was simultaneously a descent into the gritty, unglamorous details of systems engineering.

The “state management” that became a buzzword in this period was, in practice, a set of solutions to a previously overlooked problem: how to maintain context across a series of potentially failing, non-deterministic steps. Early agentic experiments often stored intermediate results in simple Python variables, which vanished if the script crashed or was restarted. Production systems demanded durable storage, versioning, and the ability to replay or branch from any point in a workflow.

This need birthed a new category of integration: the “vector memory database” faded from prominence, replaced by more conventional, transactional databases that could store not just embeddings but entire execution graphs, tool outputs, and user approvals. The harness was becoming a middleware layer, sitting between the user’s intent and a potentially sprawling, multi-model, multi-tool execution environment, and its most valuable output was often not the final answer, but the immutable audit log of how that answer was reached.

Human-in-the-loop design ceased to be a theoretical concern and became a core engineering spec. Orchestration platforms baked in “checkpoint” nodes where a workflow would pause and send a Slack message, an email, or a ticket to a human for review or guidance. This was not merely a safety feature for high-stakes tasks; it was a pragmatic admission that for many complex processes, full automation was a fool’s errand. The most effective systems were hybrid, leveraging AI for scale and speed at certain steps, and human judgment for ambiguity, creativity, or final approval at others. By formalizing these handoff points, the platforms were effectively defining the protocol by which humans and agents would collaborate. They were building the supervisory control layer for a new class of hybrid workforce, and in doing so, they were making the abstract promise of “AI agents” concrete, billable, and manageable for enterprises.

While orchestration platforms climbed to manage the process, a concurrent and equally critical battle was opening over the interface.

If agents were to reliably interact with the world—databases, software APIs, physical devices—they needed a standardized way to discover, describe, and invoke those capabilities. The question of the agent-tool interface, once fudged by framework-specific code, now demanded a universal answer.

This was the third movement of the protocol turn, and it was exemplified by the December 2023 release of Anthropic’s Model Context Protocol (MCP). MCP was not merely a technical specification; it was a strategic gambit. Anthropic proposed a server-client architecture where any resource—a SQL database, a calendar API, a file system—could be exposed to an AI model as a set of declared “tools” via a standardized JSON-over-STDIO or HTTP protocol. The model, armed with these declarations, could then request actions. Crucially, the MCP server maintained control and security; it was the gatekeeper that decided how to fulfill a model’s request for “read the quarterly_sales. csv file.”

The philosophical stance of MCP was clear: tools were outside the model’s trust boundary. The model suggested actions; the runtime environment executed them. This clean separation was a direct repudiation of the more entangled, often insecure practices of the framework era, where API keys and database connections might be bundled into a single Python process with the agent code. Anthropic was offering to standardize the fence between the AI’s reasoning and the world’s execution. By open-sourcing the protocol and providing reference servers, they were attempting to establish a de facto standard from a position of model-vendor strength. Their goal was to make their Claude models the most secure and seamlessly integrated choice for any developer building tool-using applications, thereby locking in developer mindshare and enterprise adoption through superior protocol design, not just model quality.

The reaction from the open-source and broader developer community was both swift and fractious. While many applauded the clarity and security model of MCP, others saw it as a vendor-led attempt to control the infrastructure of agentic computing. Competing proposals sprouted within weeks. A consortium of open-source projects and smaller startups floated the “Open Agent Initiative” (OAI), which advocated for a more decentralized, peer-to-peer tool discovery model inspired by web protocols. Their critique centered on lock-in: MCP, while open in specification, naturally worked best with Claude models and encouraged a centralized hub-and-spoke architecture. The OAI vision was messier but more anarchic, imagining agents that could dynamically discover and negotiate with tool-providing peers in a network. Meanwhile, pragmatic engineers in large tech companies began internal work on proprietary “tool gateway” standards, unwilling to cede control of their internal services to an external protocol, however well-designed.

This skirmish over protocols was, in essence, a debate over the power dynamics of the nascent harness layer. Would the interface between agents and the world be a neutral, open standard, or would it be a competitive moat controlled by a leading model provider? The debate echoed earlier platform wars in computing history. The outcome would determine whether the harness would evolve into a fragmented landscape of incompatible tooling or a cohesive layer that enabled true interoperability. For developers, it created a painful short-term uncertainty: investing in building an MCP server for one’s internal tools was a significant commitment, with no guarantee it would become the industry standard. This uncertainty, however, did not stall progress; it accelerated experimentation, as multiple groups raced to prove their vision of the agent-tool interface was the most powerful, secure, or flexible.

The pressure of this protocol competition fed back directly into the evolution of the model APIs themselves. OpenAI, observing the traction of Anthropic’s MCP, began quietly expanding its own function-calling ecosystem. In early 2024, it introduced more formalized “tool schemas” and enhanced the model’s ability to reason over larger, more complex sets of tool definitions. This was not a wholesale adoption of MCP, but a tactical move to ensure its models remained competitive in structured tool-use scenarios. The model vendors were now not only absorbing capabilities from below (the framework inventions) but also responding to competitive pressure from adjacent protocol layers. The harness was becoming a three-dimensional battlefield, with innovation occurring simultaneously from the top down (orchestration), the bottom up (model-native features), and the sides in (competing interface standards).

Amidst this flurry of platform and protocol development, the fate of the early framework pioneers served as a constant, sobering case study.

LangChain’s journey from “agent OS” to “component library” was not a failure, but a forced evolution. Its team, demonstrating the agility that had fueled its initial rise, pivoted hard.

They began deprecating parts of their most complex abstraction trees and instead focused on becoming the best provider of those indispensable connectors and utilities that even the thinnest wrapper or the newest orchestration platform would need. They launched “LangSmith,” a commercial observability and debugging platform for AI applications, acknowledging that as systems grew more complex, the need to trace, monitor, and evaluate LLM calls became a critical—and billable—problem.

They were climbing, too, away from the absorbed layer of core agent logic and into the adjacent layer of developer tooling and observability. Their story was a microcosm of the entire ecosystem’s motion: vertical integration by the model vendors created lateral opportunities and forced upward mobility for everyone else.

This period solidified a new, more nuanced understanding of the “harness” itself. It was no longer a single piece of software or a framework.

It was now a stack, composed of several distinct but interconnected strata. At the base, the model APIs with their native tool-calling and structured output. Above that, the protocol layer defining the agent-tool interface—be it MCP, an open alternative, or a proprietary standard. Sitting atop that, the orchestration and state-management layer responsible for workflows, durability, and human integration. And wrapping it all, the observability and evaluation tools needed to manage this new software paradigm in production.

Each of these layers was now a site of furious innovation, competition, and startup formation. The scaffolding paradox had not eliminated the harness business; it had fractalized it, creating a multitude of new problems to solve at higher levels of abstraction, each one a step further removed from direct model competition and a step closer to the messy reality of enterprise IT and global software infrastructure.

</think>To understand the velocity of this migration, one must examine the developer conversations that unfolded in the final months of 2023. On Hacker News threads and in specialized Discord servers, a palpable frustration with framework overhead began to crystallize into a new best practice.

A widely shared blog post from a senior engineer at a mid-sized SaaS company, titled “Why We Ripped Out LangChain,” became a canonical text. It meticulously documented the latency introduced by nested abstractions, the opaque error messages that made debugging a nightmare, and the startling realization that over eighty percent of their LangChain code was merely configuring objects to do what the OpenAI API could now do natively with a few lines of structured prompting. The post concluded not with a rejection of agents, but with a manifesto for simplicity: “The model is the engine. Your code should be the steering wheel, not a second, worse engine built around it.”

This sentiment resonated precisely because it captured the post-paradox shift: developers now saw the model vendors as the providers of the core cognitive engine, and their own role as integrators and conductors of that engine’s capabilities.

This technical reassessment collided with a shifting financial narrative. LangChain’s previous valuation had been predicated on its potential to become the dominant “agent OS,” a platform that would capture value from every application built on top of it, much like an operating system captures value from the software ecosystem it hosts. However, as native tool use demonstrated that the most critical “OS-level” functions—process scheduling in the form of tool selection, and system calls in the form of structured output—were being internalized by the “hardware” (the model itself), that narrative became untenable. Venture capitalists, now scrutinizing pitches with a sharper eye for durability against model vendor expansion, began to ask pointed questions about defensibility. The result was a quiet but significant down-round in early 2024, a reset of expectations that reverberated across the harness startup landscape.

The new narrative became “picks and shovels” rather than “land grabs.” LangChain’s value was re-anchored to its utility as a rich, well-maintained collection of integrations—the “shovels” for the AI gold rush—rather than as the claim to the gold mine itself.

This unbundling had a democratizing effect. The barrier to entry for building agentic applications lowered dramatically. A solo developer could now write a simple Python script that used the direct OpenAI API for function calling, a local SQLite database for state, and a bit of custom logic for retries. This “DIY harness” approach was crude but empowering, and it fueled a new wave of experimentation at the edges.

Niche vertical applications—a bot that managed a Discord server’s calendar, a tool that proofread and formatted academic citations—sprang up not as massive startups but as weekend projects or internal productivity tools. The center of gravity for innovation was diffusing away from a single framework’s ecosystem and into the broader programming community, which began to assemble its own lightweight, composable libraries for specific tasks like PDF parsing or email sending. The harness layer was not disappearing; it was disaggregating, its components becoming smaller, more specialized, and often open-source.

The rise of orchestration platforms, therefore, was not a replacement for this grassroots experimentation, but a parallel evolution addressing its scaling limits. The DIY script that worked for a hundred users would buckle under a hundred thousand. The state stored in a local file would corrupt. The lack of audit trails would become a compliance nightmare.

Platforms like Smyth and CrewAI offered industrialization. Smyth’s approach was deeply informed by lessons from large-scale data engineering. Its workflow engine was less like a scripting environment and more like Apache Airflow for AI, treating each model call or tool use as a discrete, monitored, and rerunnable task with explicit input and output artifacts.

This appealed to data teams and platform engineers in large organizations who needed to slot AI workflows into existing CI/CD pipelines and governance frameworks. The “Metering of Trust” here was explicit: every artifact was hash-linked, every decision point could be visualized, and every workflow could be rolled back to a specific step. Smyth was selling not just automation, but accountability.

CrewAI, in contrast, tapped into a different but equally powerful metaphor: organizational design. By forcing developers to define “agents” with specific roles, goals, and backstories, it encouraged a design pattern that mirrored human collaborative structures. An agent could be a “Researcher,” another a “Writer,” and a third a “Quality Checker.” The “orchestration” was the process of delegating tasks among this crew and managing their handoffs.

This abstraction proved intuitive for product managers and business analysts who were less concerned with the technical minutiae of API calls and more focused on modeling business processes. It made the “black box” of an AI system more legible by anthropomorphizing its components.

The trust mechanism here was one of role-based expectations and verification; you could define that the “Writer” agent’s output must always be reviewed by the “Quality Checker” agent before proceeding, embedding a synthetic form of peer review into the digital assembly line. Both approaches, the data-pipeline and the agent-team, were valid answers to the same core question: how do we manage process when the reasoning is commoditized?

This industrializing turn brought the harness layer into direct conversation with decades of established software engineering discipline. Concepts like idempotency—ensuring a task can be safely retried—became paramount. A non-idempotent agent instruction like “send the user a password reset email” could, if retried after a transient network error, result in a user receiving a dozen emails. Orchestration platforms had to design for this, implementing patterns like step idempotency keys or human confirmation for irreversible actions. Similarly, the field grappled with “eventual consistency” in AI-driven workflows. If two parallel agents were researching a topic, their findings might need to be synthesized by a third; managing this merge required state reconciliation logic that was alien to the stateless, single-call patterns of the early framework era. The harness was thus maturing from a set of programming conveniences into a serious discipline requiring expertise in distributed systems, data engineering, and observability.

The human-in-the-loop features of these platforms were not mere checkboxes; they were sophisticated design challenges that revealed the practical limits of full automation. Early implementations were often simple binary gates: pause for approval.

But by early 2024, more nuanced patterns emerged. “Escalation” workflows, where an AI could flag its own low-confidence output for human review, became common. “Delegate-and-verify” patterns, where a human defined the high-level plan and the AI executed the steps, reporting back at each milestone, gained traction in content moderation and legal document review. Some platforms experimented with “confidence thresholds,” allowing developers to set a minimum model confidence score below which a task would automatically route to a human.

These were more than features; they were the empirical discoveries of a field learning where silicon and carbon each excelled. They constituted the early grammar of human-AI collaboration, a grammar that was being codified not in academic papers, but in the configuration YAML files and dashboard settings of commercial orchestration products.

Amidst this progress on orchestration, the protocol battle over the agent-tool interface represented a strategic front with even higher long-term stakes. Anthropic’s launch of the Model Context Protocol was timed to exploit the industry’s acute need for a standard, arriving just as developers were drowning in the chaos of bespoke tool integrations.

MCP’s elegance lay in its separation of concerns: the model (the “client”) reasoned about what to do, and the MCP server (the “runtime”) handled the how and the if. This allowed security policies, access controls, and actual implementation details to be kept entirely outside the model’s purview and its prompt context. A bank could expose a “get_account_balance” tool via an MCP server that enforced rigorous authentication and logging, while the Claude model only needed to know the tool’s description and JSON schema.

This made MCP immediately attractive to highly regulated industries and security-conscious enterprises, providing a clear, auditable boundary between the potentially unpredictable model and core business systems.

However, the open-source and developer-led backlash was philosophically significant. Critics argued that MCP’s client-server model, with the model vendor’s AI as the privileged client, inherently centralised power. It made Anthropic’s models the natural, most compatible “brain” for any tool built on the protocol.

The competing Open Agent Initiative proposed a vision where tools and agents were more symmetrically defined peers on a network, communicating via a common protocol. In this world, an “agent” could be a specialized model fine-tuned for a task, not just a general-purpose model from a major vendor, and it could discover and use tools dynamically. This was a more radical, decentralized vision of an agentic web, akin to the early ideals of the internet itself. While technically more challenging and initially less secure, it appealed to a faction that feared the consolidation of AI power into a few vertically integrated giants controlling the models, the tool interfaces, and eventually, the orchestrators.

The practical consequence of this skirmish was a period of intense experimentation and protocol pluralism. Developers building tools in early 2024 faced a confusing array of choices: adopt MCP and bet on Anthropic’s ecosystem, implement a simpler, custom REST API for tools, or try to support multiple standards.

This fragmentation, while inefficient, was a sign of a healthy, competitive exploration of the design space. It also forced the major players to iterate rapidly. Anthropic expanded MCP’s scope, adding support for “read-only” tools and streaming responses.

OpenAI, while avoiding a direct protocol war, strengthened its own tool-use ecosystem with improved schema validation and better error handling in its API, effectively making its walled garden more hospitable. The open-source community produced reference implementations and bridges between different standards. This competitive ferment ensured that the eventual standard—or standards—that emerged would be stress-tested from multiple angles, incorporating lessons from security, scalability, and developer ergonomics.

The collective motion of these three strands—the disaggregation of the framework, the industrialization of orchestration, and the battle over the tool interface—defined the Protocol Turn.

It was a period where the industry collectively stopped trying to wrap the model and started building the world the model would interact with. The scaffolding paradox had performed its clarifying function: by absorbing the initial, obvious abstractions, the model vendors had forced the ecosystem to confront harder, more structural problems.

The value was no longer in making a single AI call work; it was in making a million AI calls work together reliably, securely, and accountably across countless real-world tasks. The harness layer was solidifying not as a monolithic platform, but as a stratified stack of interoperable services, protocols, and tools.

It was becoming, in a very real sense, the middleware of intelligence—the indispensable plumbing that would connect the raw cognitive power of the models to the vast, intricate, and unforgiving terrain of human software and business processes. The climb was arduous, but the view from this higher ledge revealed not a diminished landscape, but a far more complex and consequential one. The race was no longer to build the best prompt chain; it was to define the rules of engagement for a new era of automated, semi-autonomous software.

The sprawling abstraction tree—the complex inheritance hierarchies and the sometimes-opaque orchestration logic—began to look like a liability. It added cognitive overhead and hidden failure modes.

A new pattern emerged in forums and code reviews: developers championing “thin wrappers” or even direct API calls. They argued that wrapping the model’s native function-calling in another layer of abstraction was not just redundant but dangerous; it obscured the model’s actual behavior and made debugging harder.

The valuation narrative shifted under this technical reassessment. LangChain was no longer the inevitable operating system. It was recast, both by the market and by its own evolving positioning, as a “component library”—a useful collection of connectors, memory implementations, and pre-built chains, but not the sovereign layer. Its centrality eroded not through a catastrophic collapse, but through a thousand small decisions by developers to bypass its heaviest abstractions in favor of simpler, more direct control. The harness layer was forming, but it was refusing to coalesce around a single framework’s vision.

Parallel to this unbundling of the framework, a second movement gained momentum: the rise of explicit orchestration platforms. If the model was becoming a more reliable node—a worker that could natively use tools—then the real challenge shifted upward. It was no longer about making the model *capable.