Observability and Analytics: You Can't Deploy an Agent You Can't See

I wrote recently that the real value in AI isn't the thing that answers you — it's the thing that does the work. I stand by it. But "does the work" cuts both ways. An agent that can act is an agent that can act wrongly — at scale, without stopping to ask. Which raises the question that quietly decides whether you can put one into a real business at all: when it acts, can you see what it did?

A wrong answer is visible. A wrong action is not.

Here is what actually changed when AI stopped merely answering and started acting.

When AI only hands you an answer, the damage from a bad one is visible and contained. A person reads it, senses it's off, and doesn't act on it. The human is still the checkpoint, so the mistake dies there.

When an agent takes the action itself, that checkpoint disappears. A bad decision is now invisible and compounding. It reprices a quote, reroutes a shipment, approves a claim — and then it does the same thing again a thousand times before anyone notices, because the entire point of an agent is that there is no human in the middle reading each one. That absence is the value. It is also the risk, and it is the same absence. 

Picture a claims agent that starts misreading a single field on one policy type and quietly auto-approves a category of claims it never should have. Nothing looks broken. No error is thrown. The dashboard stays green. You find out at the quarterly review, by which point the number has four zeros on it. That is what a wrong agent looks like in practice — not a crash, but a silent, confident, repeated mistake that no one was positioned to catch.

This is why so many agent projects that dazzle in a demo get walked back within a month. Not because the agent got dumber. Because the moment it touched real work, nobody could tell whether it was doing the right thing — and in a serious business, "we're not sure what it's doing" is a reason to switch it off.

Observability for an agent is not a green dashboard

When people hear "monitoring," they picture a dashboard that says the service is up. For an agent, that is almost useless. "The system is running" tells you nothing about whether the agent is making good decisions.

What you actually need is closer to a flight recorder. For every single run, the full trace: every decision the agent made, every tool it called, every step from the request coming in to the action going out — and each of those steps broken down into its own parts, so you can see not just that it acted, but how it got there. In the language of the field, that is traces and spans: the complete execution path, and its nested detail.

And you need to measure things classic software never had to. Not only latency and cost, but whether the agent's answer was actually grounded in real data or quietly invented — what's often called faithfulness. Whether it reached for the right tool for the job. Where users give up. Where a multi-step flow breaks, and at which step. You are not watching whether the agent is running. You are watching whether it is reasoning.

The three questions it has to answer

Strip away the tooling, and observability for agents has to answer three questions, in order.

First: is the fleet healthy? Across every agent you are running — success and failure rates, latency, cost. Which agents are thriving and which are degrading, ideally before a customer is the one who tells you.

Second: where exactly did this one break? When something fails, you need to land on the precise step. Was it the model, the retrieval, a misfiring tool, or a downstream system from 2009 that timed out? "The agent failed" is a shrug. "The agent failed at the tool call because the ERP didn't respond" is something an engineer can fix by lunch.

Third, and hardest: did it actually finish the real job? Not "did it produce a response," but "did the real-world task complete." 

Outcome-level visibility — did the shipment actually move, did the claim actually close — is what separates a demo metric from a number the business cares about.

You test software before you ship it. Agents, mostly not.

Everything so far is about watching an agent once it is live. But the strongest teams don't wait for production to learn whether an agent works. You would never ship software without running it against a test suite first. Somehow, agents get pushed live on the strength of a slick demo and a hope.

Evaluation is the other half of AgentOps, and it happens before anything reaches production: stress-testing the agent against a battery of realistic cases — does it stay grounded, does it call the right tools, does it hold up on the messy edge cases and not just the happy path someone rehearsed for the demo. At most companies this is still done by hand, occasionally, and badly. It shouldn't be. An agent that hasn't been run against hundreds of real scenarios isn't production-ready; it is a demo that simply hasn't failed yet. Observability tells you what an agent did. Evaluation tells you what it is going to do — before you find out the expensive way.

In a regulated business, observability is the permission slip

Now push this into a bank, an airline, an insurer, and it stops being an operational nicety and becomes the whole ballgame.

In those environments, "the agent did the right thing" is not enough. You have to be able to prove it — to an auditor, to a regulator, to a customer — after the fact, sometimes months later. Every action the agent took has to be reconstructable and defensible on demand. That is not something you add for polish. It is the condition of being allowed to run at all.

Put plainly: in a regulated industry, an agent you cannot audit is an agent you are not permitted to deploy, no matter how good it is. This is the part that gets missed in every "look what our agent can do" demo. Observability is not the thing you go check when something breaks. It is the permission slip that lets the agent act in the first place — the thing that turns autonomy from a liability your risk committee vetoes into something they will actually sign.

AgentOps is a discipline, not a purchase

There is a temptation to treat all of this as a box to tick — buy the monitoring tool, done. That misreads what it is.

Think about what DevOps actually did for software. It didn't make software better by adding a product. It made shipping continuous and safe by turning it into a practice: observe what is happening in production, catch problems early, fix, repeat, forever. AgentOps is that same discipline pointed at agents — observe what the agent does, evaluate whether it was right, optimize, and keep doing it while the thing is live. You don't buy it once and move on. It is how you run agents, the way DevOps is how you run software. Teams that understand this build the observability in first. Teams that don't discover they needed it in front of a customer.

What I see, and why we built it in

The pattern is consistent. The teams that put agents into real production treat observability as day-zero infrastructure — the first thing they wire in, before the agent is allowed anywhere near a live process — not a day-100 scramble after something goes wrong publicly. The ones who bolt an agent into production and hope are, reliably, the ones quietly pulling it back out a few weeks later.

That is exactly why we built observability into Mashu AI Orchestrate as a first-class capability rather than an afterthought. Full traces for every agent, fleet-level health across all of them, evaluation to stress-test agents before they go anywhere near a live process, and a governance layer that makes what an agent did provable after the fact — so a regulated enterprise can let an agent act and still answer cleanly to its auditor. We built it that way because none of the above is theory to us. It is the difference between the agents that stay in production and the ones that don't.

The agent acts — and you can prove it was right

The agent acts. That was the whole promise, and it is a real one. But in any business that has to answer for what it does, "the agent acts" is only half a sentence. The other half is "and we can see exactly what it did, and prove it was right."

One half without the other gives you one of two things: a toy, or a time bomb. The companies that will actually run agents — not demo them, run them, in businesses where a wrong move has consequences — are the ones treating that second half as seriously as the first. That is the entire difference between an agent you show off and an agent you can trust with the business.

I know which one is worth building.