CrocopliersCrocopliers

Engineering Perspective

AI Agents in Practice: What Actually Works Beyond the Demo

AI agents are the technology that every conference talks about and every engineering team struggles to ship. The demos are compelling: an agent that researches, plans, executes, and reports — all without human intervention. In production, the reality is different. Agents fail in ways that are hard to predict, expensive to debug, and embarrassing to explain to users. But in specific, well-bounded domains, they are genuinely transformative. The difference is not the model — it is the engineering around it.

9 min read2026-11-13AI Engineering

What people mean when they say agent

The term agent is used for everything from a chatbot with a system prompt to a fully autonomous system that takes actions across multiple APIs. For this discussion, an AI agent is a system where a language model decides what to do next — it chooses which tools to call, what data to retrieve, and when to stop. The human provides a goal, not a script.

This is fundamentally different from a pipeline where the developer defines the sequence. In a pipeline, you control the flow. With an agent, the model controls the flow. That single difference is what makes agents both powerful and dangerous. When the model makes good decisions, the agent handles novel situations gracefully. When it makes bad decisions, it can take expensive, irreversible actions before anyone notices.

Where agents deliver real value today

Agents work best in domains with three properties: well-defined tools with predictable outcomes, a clear success criterion the agent can evaluate itself, and bounded scope where a wrong decision is recoverable. The strongest production use cases right now are internal operations — not customer-facing products.

Code generation and development assistance is the most mature agent use case. The tools are well-defined (file read, edit, run tests, search), success is verifiable (tests pass, code compiles), and failures are cheap to undo (git revert). This is why coding agents have moved fastest from demo to daily use.

Document processing and data extraction is another strong domain. An agent that reads invoices, extracts structured data, cross-references against existing records, and flags anomalies. The tools are clear, the success criteria are measurable, and errors are caught by downstream validation. Customer support triage — reading a ticket, classifying it, pulling relevant context, and drafting a response for human review — follows the same pattern.

  • Code generation: well-defined tools, verifiable outcomes, cheap failures
  • Document processing: structured extraction with downstream validation
  • Support triage: classification + context retrieval + human-reviewed drafts
  • Internal automation: data reconciliation, report generation, system monitoring

Why most agent projects fail after the pilot

The pilot works because the person who built it is watching. They chose good examples, tuned the prompts for those examples, and manually handled the edge cases. In production, the agent encounters inputs the builder never imagined, calls tools in sequences nobody anticipated, and fails in ways that are hard to reproduce.

The core problem is compounding errors. In a three-step agent workflow, if each step is 95% reliable, the end-to-end reliability is 85%. For a five-step workflow, it drops to 77%. For ten steps, 60%. Every additional step multiplies failure probability. This is why agents that look great in demos — where the happy path has five steps — fall apart in production where real tasks have fifteen steps with conditional branches.

The second killer is cost. Agent loops call the model multiple times per task. A task that costs $0.50 in API calls and takes 30 seconds sounds reasonable until you realize 20% of runs spin into retry loops that cost $5 and take 3 minutes before timing out. Without hard stops on loop count, token budget, and wall-clock time, agent costs are unbounded.

How to build agents that survive production

The key insight is that production agents need more engineering around the model, not a better model. Start with hard limits: maximum number of tool calls per task, maximum token budget, maximum wall-clock time. When any limit is hit, the agent stops and asks for human help instead of spinning. This is not a workaround — it is the design.

Build observability from day one. Log every model call, every tool invocation, every decision. You need to replay any agent run and understand exactly why it did what it did. When an agent makes a bad decision — and it will — the question is not what happened but why did the model choose this step, and how do we prevent it.

Use the cheapest model that works. Agent loops multiply model costs by the number of steps. A model that is 30% cheaper per call saves 30% on a single call but saves 80% on a ten-step agent run that would otherwise hit your cost ceiling and fail. Fast, cheap models for tool selection and routing; expensive models only for complex reasoning steps.

Design every tool call to be idempotent and reversible where possible. If an agent creates a record, it should be able to delete it. If it sends a message, the message should be a draft until a human approves. The less irreversible each action is, the safer the agent is to run autonomously.

The human-in-the-loop question

The right amount of human oversight depends on the cost of a mistake. For internal data processing where errors are caught by downstream systems, full autonomy is fine. For customer-facing actions — sending emails, modifying accounts, processing payments — a human should approve the action before it executes.

The common mistake is putting the human in the middle of the loop instead of at the approval gate. An agent that stops and asks after every step is not useful — it is a chatbot with extra steps. The human should review the agent's plan or its final action, not micromanage the execution. Think of it as code review, not pair programming.

Over time, as you build confidence in the agent's decisions through logging and auditing, you can widen the autonomy boundary. Start narrow, prove reliability with data, expand gradually. This is the same pattern as any production system — canary deployments, not big-bang launches.

How to evaluate whether your use case is agent-ready

Before building an agent, run a simple test. Have a person do the task ten times while writing down every decision they make and every system they touch. If the list of decisions is under ten, the tools are under five, and the success criteria fit in one sentence — an agent can probably handle it. If the person cannot describe their decision process because it involves judgment, context, and experience — the task is not agent-ready.

The best agent candidates are tasks that knowledge workers do reluctantly: tedious, well-defined work that requires touching multiple systems but not deep reasoning. Data reconciliation, status reporting, compliance documentation, onboarding checklists. Nobody loves doing these tasks, the rules are clear, and automation directly improves the team's quality of life.

FAQ

Common questions

What is the difference between an AI agent and a chatbot?
A chatbot responds to messages. An agent takes actions. A chatbot answers your question about an order status. An agent checks the order system, sees it is delayed, drafts an updated delivery notice, and queues it for your approval. The core difference is that the agent decides which tools to use and in what order, rather than following a fixed script.
Are AI agents reliable enough for production use?
In bounded domains with well-defined tools and verifiable outcomes — yes, with appropriate guardrails. The key is limiting scope, setting hard stops on execution loops, and keeping a human in the approval path for irreversible actions. Agents are not reliable enough for open-ended tasks with unpredictable tools and no validation. The reliability question is always about the specific use case, not agents in general.
How much does it cost to run AI agents in production?
Agent costs depend on the number of model calls per task. A simple three-step agent using a fast model might cost $0.01-0.05 per task. A complex ten-step agent using a reasoning model can cost $0.50-2.00 per task. The risk is unbounded loops — always set maximum step counts and token budgets. Start by measuring the cost of ten representative tasks, multiply by expected daily volume, and add a 3x buffer for retries and edge cases.

Next step

Considering AI agents for your product or operations?

We build production AI agent systems — scoped, instrumented, and designed to survive real-world conditions. Tell us about the task you want to automate and we will assess whether an agent is the right approach.

Book a project assessment