AI Coding Agents: What They Are and How They Actually Work

AI Coding Agents: What They Are and How They Actually Work

The difference between an AI coding agent and autocomplete isn't speed or model quality. It's who holds the loop. Autocomplete predicts the next few tokens and hands control back to you. An agent keeps control: it decides what to read, what to change, what to run, and what to do next based on what happened, without you approving each step.

That distinction sounds small. It's the entire reason agents can do things autocomplete never could: open a failing test, read the stack trace, patch the code, rerun the test, and stop when it's green. It's also the reason they fail in ways autocomplete never does, confidently, three files deep into a change nobody asked for.

This guide is for engineers deciding whether an agent belongs in their workflow, not for people picking a favorite. It skips the vendor tour. What follows is the actual mechanism: the loop, the tools, the context problem nobody puts on the landing page, and a way to evaluate any agent against your own codebase rather than against a marketing page.

What makes something an agent, mechanically

Anthropic's engineering team, who ship one of the more widely used coding agents, draw the line this way: a workflow is "a system where LLMs and tools are orchestrated through predefined code paths," while an agent is a system "where LLMs dynamically direct their own processes and tool usage" [1]. Autocomplete and even most chat-based code generation are workflows in this sense: you author the path (accept this suggestion, paste this snippet, run this command yourself). An agent authors the path.

Under the hood, that architecture is not exotic. The same engineering post describes it as three ingredients: an environment the agent can act on, a set of tools that let it act, and a system prompt defining its goal and constraints, wired to a model that runs "in a loop." That's the whole design in its simplest form [1]. The idea traces back further, to a 2022 research paper called ReAct. It showed that letting a language model interleave reasoning ("I should check whether this function is even called anywhere") with acting (actually running a search) beat models that only reasoned or only acted, because the reasoning traces let the model track a plan and recover when something didn't go as expected [2]. Claude Code, GitHub Copilot's agent mode, and Cursor's Agent can each reasonably be read as a productized, code-specific descendant of that pattern; none of them cite it directly in their own marketing, but the reason-then-act structure is the same one.

The loop, worked through concretely

Here's what actually happens, step by step, when you tell an agent "the test_checkout_total test is failing, fix it":

1. The model gets your instruction plus its system context: the repo structure, relevant tool definitions, maybe a memory file describing project conventions.
2. It decides an action is needed and calls a tool. Concretely, this means the model's response includes a structured tool-call block (something like "run pytest tests/test_checkout.py -k test_checkout_total") rather than freeform text. Anthropic's API implements this literally: a tool-use block naming which tool and with what arguments [1].
3. The environment executes that action and returns ground truth: in this case, the actual test output and stack trace. This step is what separates an agent from a model just guessing: it doesn't proceed on its own assumption of what the code does, it reads what actually happened [1].
4. The model reasons over the result and decides the next action: open checkout.py, search for where total is computed, read the surrounding 40 lines.
5. It proposes an edit, which, depending on the product, either applies automatically or waits for your approval (more on that split below).
6. It reruns the test. If it passes, the loop can stop. If not, it reads the new failure and goes again.
7. A stopping condition eventually ends the loop: task complete, a maximum iteration count, or a point where the agent decides it needs your input. Anthropic's own guidance is explicit that production agents need these limits to "maintain control"; an agent without a stopping condition is a liability, not a feature [1].

Steps 2 through 6 repeat until the task resolves or the loop bails out. Nothing about this requires a giant model or a novel algorithm. What makes one agent meaningfully better than another is almost entirely in the engineering around that loop: how good its tools are, how well it manages what's in context at each turn, and how it decides when to ask you versus act alone.

How they read, edit, and run code

All three mainstream products expose roughly the same tool categories to the model (read files, search the codebase, propose or apply edits, run terminal commands), but they differ on where the human sits in the loop, which is the decision that actually matters for a team:

• GitHub Copilot's agent mode (shipped in preview February 2025) builds "a summarized structure of the workspace" rather than dumping the whole repo into context, then lets the model call tools for file reads, workspace search, terminal execution, and compiler/linter errors. Terminal commands specifically require your approval before they run, and every tool call is shown in the UI as it happens [3]. That's a deliberate autonomy ceiling: the model can read and search freely, but executing anything is gated.
• Cursor's Agent reads files, searches the codebase and the web, edits and applies changes, and executes terminal commands. Its documentation describes a checkpoint system for reviewing and reverting changes after they're made, but, unlike Copilot's docs, it doesn't spell out a pre-execution approval gate the same way [4]. Read that as a documented recovery mechanism, not a confirmed claim that nothing ever requires upfront approval; check the current docs before rolling out to a team.
• Claude Code uses a tiered permission system: read-only actions (file reads, searches) don't require approval within the working directory, while Bash commands require approval except for a built-in set of read-only commands, and each rule can be checked into version control so a whole team shares the same policy rather than each developer approving from scratch [5].

None of this is a claim about which is "better." It's a claim about what you're actually agreeing to when you turn one of these on: how much can it do before a human sees it, and how expensive is undoing it if it's wrong. That's a fact worth pulling directly from a product's own docs before rollout, not inferring from a demo video.

The problem no landing page mentions: context rot

Every one of these tools is bounded by a context window, and the naive fix, "just use a bigger window," doesn't actually solve the problem it looks like it solves. Anthropic's engineering team, describing why they built explicit context-management systems into their agent products, put it plainly: "as the number of tokens in the context window increases, the model's ability to accurately recall information from that context decreases." That pattern is what they call context rot, and it happens even when there's technically room left in the window, because of how attention scales across a longer sequence [6].

For a coding agent, this is not an abstract concern. It's the mechanism behind the failure mode every engineer who's used one of these tools has seen: the agent forgets a constraint you stated forty tool calls ago, or re-introduces a bug it already fixed, or starts contradicting a decision it made two files back. The tools that handle this well use two techniques, both described directly in Anthropic's own engineering writing:

• Compaction: when a session nears its context limit, the agent summarizes the conversation so far and reinitializes with the compressed version instead of the raw history. The failure mode here is real too: summarize too aggressively and a detail that mattered later gets lost [6].
• Subagents: rather than one long-running context accumulating everything, a coordinating agent delegates a focused task to a subagent that explores in its own clean context (potentially tens of thousands of tokens of searching and reading) and returns only a condensed result, often on the order of one or two thousand tokens, back to the parent [6].

If you're evaluating an agent for a large, old codebase (the kind where "find every place this deprecated function is called" genuinely requires reading dozens of files), this is the question to ask a vendor, not "how big is your context window." A bigger window with no compaction or subagent strategy degrades the same way; it just takes longer to get there.

How agents reach outside your editor: MCP

A coding agent is only as useful as the systems it can touch, and until late 2024 every one of those connections was custom-built: a bespoke integration for your issue tracker, another for your CI logs, another for your internal API docs. Anthropic introduced the Model Context Protocol (MCP) in November 2024 as an open standard meant to collapse that: instead of "every new data source requires its own custom implementation," a tool exposes itself once as an MCP server, and any MCP-compatible agent can use it as a client [7]. Anthropic has described the problem this solves as an M×N one: M different agent products, N different tools each of them might need to connect to, previously meaning M×N separate integrations built and maintained by someone [7]. Whether an agent supports MCP, and how many servers already exist for the systems your team runs on, is a fair adoption question: it determines whether "connect this to our internal deploy tool" is a config change or an engineering project.

Evaluating an agent for your team, not for a demo

A demo answers "can this write a CRUD endpoint." That's not the question that determines whether an agent is worth adopting on a real codebase with a real on-call rotation. A more useful evaluation runs through the mechanism this guide just walked through:

• What's the permission model, concretely, not marketingly? Does it gate execution before it happens (Copilot's terminal-approval model) or let you revert after (Cursor's checkpoint model)? Neither is wrong, but they imply different review habits, and a team that assumes pre-execution gating when the tool actually does post-hoc checkpoints will get surprised.
• What happens on your context, not a toy repo? A 40,000-line service and a 4-file starter project stress context management completely differently. Ask specifically how the tool handles a task that requires touching many files: does it degrade quietly (context rot, confidently wrong answers), or does it have an explicit compaction or subagent strategy you can point to in its own documentation?
• Is the loop actually verifiable? The agentic loop works because each step gets ground truth: a test result, a build error, a lint failure. An agent working against a codebase with no tests, no CI, or a flaky test suite loses that feedback signal, and the loop degrades into an LLM guessing repeatedly rather than genuinely iterating. If your test suite isn't trustworthy, fix that before evaluating agents, not after.
• What's the actual audit trail? Every tool call visible in real time (Copilot's stated UI behavior), or a diff you review after the fact? For a regulated codebase or anything with change-control requirements, this is not a preference, it's a compliance question.
• Does it support the systems you actually run? MCP support tells you whether connecting the agent to your internal tools is a configuration exercise or a build project.

None of these questions have a universally correct answer. They have an answer that's correct for your codebase, your test coverage, and how much risk your team is willing to hand to something that runs in a loop without asking permission at every step.

What this doesn't tell you

This guide is deliberately conceptual. It explains the mechanism these tools share, not how any specific product performs on any specific task. It makes no claim about which agent is fastest, most accurate, or best at any language or framework; that requires hands-on testing against real work, which is a different kind of article. It also can't tell you whether the tool works well on codebases with unusual build systems, monorepo tooling, or heavily customized CI; those are things you verify against your own repository, not against a guide.

The mechanism, though, doesn't change much between products, and understanding it is what turns "we tried an AI coding agent and it was weird" into a specific, answerable question: was it a permission-model mismatch, a context-management failure on a large task, or a broken feedback loop from an untrustworthy test suite. Most agent disappointments trace to one of those three.

---

Sources

1. Anthropic, "Building Effective Agents", Anthropic Engineering.
2. Yao et al., "ReAct: Synergizing Reasoning and Acting in Language Models", arXiv:2210.03629, ICLR 2023.
3. "Introducing GitHub Copilot agent mode (preview)", Visual Studio Code Blog, February 24, 2025.
4. "Agent Overview", Cursor Docs.
5. "Configure permissions", Claude Code Docs.
6. Anthropic, "Effective Context Engineering for AI Agents", Anthropic Engineering.
7. Anthropic, "Introducing the Model Context Protocol", Anthropic News, November 25, 2024.